IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 



In re application of: Vincent Fedele 



Attorney Docket No.: IDTXP052B 



Application No.: 10/561,647 



Examiner: Unknown 



Filed: June 21, 2004 



Group: Unknown 



Title: ACQUISITION OF HIGH RESOLUTION 
BIOMETRIC IMAGES 



Confirmation No.: 7722 



CERTIFICATE OF EFS-WEB TRANSMISSION 



I hereby certify that this correspondence is being transmitted electronically 
through EFS-WEB to the Commissioner for Patents. P.O. Box 1450 Alexandria. 
VA 22313-1450 on July 21, 2008. 



Signed: 



/Tara Hayden / 



Tara Hayde 



PETITION FOR APPLICATION BY OTHER THAN ALL THE INVENTORS 
Pursuant to 37 C.F.R. §1.47 (a) 



Commissioner of Patents 

P.O. Box 1450 

Alexandria, VA 22313-1450 



Applicants John S. Berg, Christopher Butler, and Robert David Farraher (the 
"remaining joint inventors") hereby petition the Commissioner to accept for application the 
above-identified U.S. Patent Application by other than all of the inventors. 

The Declaration For Patent Application is attached in which the remaining joint inventors 
have signed on behalf of the omitted inventors as required by 37 C.F.R. §1.64 and M.P.E.P. 
§409. 03(a), and it includes the last known address of the omitted inventor. 

A Declaration of Facts in Support of Applying on Behalf of the Omitted Inventor is also 
attached and provides proof of the pertinent facts regarding the omitted inventor who refuses to 
sign as required by 37 C.F.R. § 1.47(a) and M.P.E.P. §409.03(d). 

In view of the refusal of the omitted inventor to sign the papers required for the above- 
identified application, the remaining joint inventors are believed to be entitled to make such an 



Dear Sir: 



1 



application on behalf of and as agents for the omitted inventor. The required fee pursuant to 37 
C.F.R. § 1.17(g) is enclosed. 



Dated: July 21. 2008 Respectfully submitted, 

WEAVER AUSTIN VILLENEUVE & SAMPSON LLP 

/Denise S. Bergin/ 

Denise S. Bergin 
Registration No. 50,581 

P.O. Box 70250 
Oakland, CA 94612-0250 
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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 



In re application of: Vincent Fedele 



Attorney Docket No.: IDTXP052B 



Application No.: 10/561,647 



Examiner: Unknown 



Filed: June 21, 2004 



Group: Unknown 



Title: ACQUISITION OF HIGH RESOLUTION Confirmation No.: 7722 
BIOMETRIC IMAGES 



DECLARATION OF FACTS IN SUPPORT OF APPLYING ON BEHALF OF OMITTED 
INVENTOR 
Pursuant to 37 C.F.R. §1.47 (a) 



Commissioner of Patents 

P.O. Box 1450 

Alexandria, VA 22313-1450 



This declaration is made as to the facts that are relied upon to establish the diligent effort 
made to secure the execution of the Declaration for the subject U.S. application by the omitted 
inventor, Vincent Fedele. This declaration is being made by the available person having first- 
hand knowledge of the facts recited therein. 

I, Denise S. Bergin, do hereby declare: 

1. The subject U.S. application is based on a Patent Cooperation Treaty (PCT) application 
flied on June 21, 2004 and having International Application No. PCT/US2004/019713. 

2. All rights in and throughout the United States of America, its territories, and all foreign 
countries, in and to the invention described in the subject PCT application were assigned to 
Aprilis, Inc. ("Aprilis"). An Assignment was executed by all the inventors, including Vincent 
Fedele, and is was recorded in the U.S. Patent and Trademark Office (PTO). See attached 
Exhibit A. 

3. The prior law firm representing Aprilis, Hamilton, Brook, Smith & Reynolds, P.C., 
forwarded, on March 6, 2006, a Declaration For Patent Application ("Declaration") to Mr. David 
A. Waldman at Aprilis for execution by all the inventors. See attached Exhibit B. 

4. This Declaration was executed by all the inventors, except for Vincent Fedele. 
Apparently, prior to the time the Declaration was sent to Aprilis, Vincent Fedele had left Aprilis. 



Dear Sir: 



5. Some time in July 2006, responsibility for the subject U.S. application was transferred 
from Hamilton, Brook, Smith & Reynolds, P.C. to Beyer, Weaver & Thomas LLP, now Weaver, 
Austin, Villeneuve & Sampson LLP. 

6. On June 4, 2008, I sent a copy of the PCT application along with a Declaration and 
Assignment to Vincent Fedele at his last known address. These documents were sent via 
Express Mail. I asked that Mr. Fedele return the executed Declaration and Assignment by June 
20, 2008. The package was delivered on June 7, 2008, with a signature release for Mr. Fedele on 
file. See attached Exhibit C. 

7. I have not received the signed Declaration or Assignment, or any other communication 
from Mr. Fedele. 

8. Attached is a letter from Vincent Fedele dated July 27, 2006. This letter was sent in 
response to a request to execute a Declaration and Assignment for related US Application No. 
10/561,646, which shares the same filing date and priority claim as the instant application. By 
this letter, Vincent Fedele indicated that he would not sign Declarations and Assignments nor 
review applications for subject matter invented during his time at Aprilis, unless he received 
compensation from Aprilis. See attached Exhibit D. 

9. Mr. Fedele refused to sign a Declaration for related US Application No. 10/561,646. The 
remaining inventors submitted a petition pursuant to 37 C.F.R. §1.47 (a) accompanied by a 
Declaration of Facts by William J. Egan, III and including the letter from Mr. Fedele that is 
attached as Exhibit D. The petition was granted. See attached Exhibit E. 



I hereby declare that all statements made herein of my own knowledge are true and that 
all statements made on information and belief are believed to be true; and further that these 
statements were made with the knowledge that willful false statements and the like so made are 
punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United States 
Code, and that such willful false statements may jeopardize the validity of the application or any 
patent issued thereon. 



/Denise S. Bergin/ 
Denise S. Bergin 



July 21. 2008 



Date 



EXHIBIT A 
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FEBRUARY 15, 2005 

PTAS 

HAMILTON, BROOK, SMITH & REYNOLDS, P.C. 
TIMOTHY J. MEAGHER 

530 VIRGINIA ROAD, P.O. BOX 9133 
CONCORD, MA 01742-9133 



UNITED STATES DEPARTMENT OF COMMERCE 
Patent and Trademark Office 

ASSISTANT SECRETARY AND COMMISSIONER 
OF PATENTS AND TRADEMARKS 
Washington, D C, 20231 
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UNITED STATES PATENT AND TRADEMARK OFFICE 
NOTICE OF RECORDATION OF ASSIGNMENT DOCUMENT 

THE ENCLOSED DOCUMENT HAS BEEN RECORDED BY THE ASSIGNMENT DIVISION OF 
THE U.S. PATENT AND TRADEMARK OFFICE. A COMPLETE MICROFILM COPY IS 
AVAILABLE AT THE ASSIGNMENT SEARCH ROOM ON THE REEL AND FRAME NUMBER 
REFERENCED BELOW. 

PLEASE REVIEW ALL INFORMATION CONTAINED ON THIS NOTICE. THE 
INFORMATION CONTAINED ON THIS RECORDATION NOTICE REFLECTS THE DATA 
PRESENT IN THE PATENT AND TRADEMARK ASSIGNMENT SYSTEM. IF YOU SHOULD 
FIND ANY ERRORS OR HAVE QUESTIONS CONCERNING THIS NOTICE, YOU MAY 
CONTACT THE EMPLOYEE WHOSE NAME APPEARS ON THIS NOTICE AT 703-308-9723. 
PLEASE SEND REQUEST FOR CORRECTION TO: U.S. PATENT AND TRADEMARK OFFICE, 
ASSIGNMENT DIVISION, BOX ASSIGNMENTS, CG-4 , 1213 JEFFERSON DAVIS HWY, 
SUITE 320, WASHINGTON, D.C. 20231. 



RECORDATION DATE : 02/08/2005 REEL/FRAME: 015685/0553 

NUMBER OF PAGES: 5 

BRIEF: ASSIGNMENT OF ASSIGNOR'S INTEREST (SEE DOCUMENT FOR DETAILS). 
DOCKET NUMBER : 317 4.1012-00 6 



ASSIGNOR: 

FEDELE, VINCENT 

ASSIGNOR: 

BERG, JOHN S. 

ASSIGNOR: 

BUTLER, CHRISTOPHER J. 

ASSIGNOR: 

FARRAHER, ROBERT DAVID 



DOC DATE: 12/20/2004 

DOC DATE: 01/19/2005 

DOC DATE: 12/14/2004 

DOC DATE: 12/14/2004 



ASSIGNEE: 

APRILIS, INC. 

5 CLOCK TOWER PLACE 

SUITE 200 

MAYNARD, MASSACHUSETTS 01754 



o;o£ ftvjc uuo/uun rax server 



015685/0553 PAGE 2 

SERIAL NUMBER: FILING DATE: 

PATENT NUMBER: ISSUE DATE: 

PCT NUMBER: US0019713 
TITLE: 



VIOLET MCCOY, EXAMINER 
ASSIGNMENT DIVISION 
OFFICE OF PUBLIC RECORDS 



RELORDA¥t61^ptftd/ COVER SHEET ~ ~~ 

PATENTS ONLY Dock* No.: 3174.1012-006 



To The Djjtcioi of [he U.S. Patent and Trademark Office P lease record the attached documents or the new addresses) below. 


J Name of conveying partytiesj/Execuuon £>aie{s)' 

1. VINCENT FEDELE 

2. JOHN S. BERG 

3. CHRISTOPHER J. BUTLER 

4. ROBERT DAVJD FARRAHER 

Execution Daw. 11 12/20/04, 21 1/19/05, 3,4) 12/i4/fM, 


2. Narnc and address of receiving parties) 

Name: AFRILIS, INC. 


Internal Address- 




Additional namcisj of conveying paretics) »H4C&«1? [ J Yes [ X ] No 


Strsei Address: 5 CLOCK TOWER PLACE 


3. Mature of conveyance: 

[X } Assignment [ J Merger 

| j Security Agreement [ ] Change ofNamc 

I J Other 


SUITE 200 


City. MAYNARP 


State: MA 


Country USA Zip: 01754 




Aa<iaigiia|n9Bie<si&addrcss(csj«n«caed'' [ ] Yes [ X ) No 




4. Application aumber(s) or patent Humberts). 
A Patent Application No.(s) 
PCTAJS2004/019713 

Additional numners 


I ] This document is being filed togetlier with a new application. 
3 ParenrNo.(s) 

inched-/ [ ] Yes L X 1 No 


S . Name and address of party to whom correspondence 
concerning document should be mailed' 

Nmr TIMOTHY J. MEAGHER 


6. Total number of applications and patents involved: [ J ] 


7 Total Fee (37 Cf.R. 3.41). , $ 4Q 


Internal Address 


[ ] Enclosed 

[X ] Authomstioft to charge dqsosit account number 08-U38O 

[ ] Piously submitted -Doc. ID No [ ] 

ryi Authoriacd to charge any deficiencies or credit any overpayment to 
deposit account number 08-0380 

Do not attach a copy of this page if paying by deposit account and tiling 
vis facsimile. 

Attach a copy of This page if paying by deposit account ami filing via mail 


Hamilron, Brook, Smith & Reynolds, P C. 


Street Address 530 Virginia Road, P.O. Box 9133 
C.vy Concord Stare- MA ZIP 01 742-9133 


Phone No. 978-341-0036 F«*No 978-341-0136 


Email Address. 




8. MARK B. SOLOMON ^rj ^ H lg_ _ 

Ntime of Person Signing 1 " Signature Dare 

Total number of pages including cover sheet, attachments, and document. [ 5 ] 
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REC . RDATION FORM COVER SK ,.£T 

PATENTS ONLY Docket No.: 3174.1012-006 


To the Director of the U.S. Patent and Trademark Office: Please record the attached documents or the new address(es) below. 


1 . Name of conveying party(ies)/Execution Date(s) : 

1. VINCENT FEDELE 
! 2. JOHN S. BERG 

3. CHRISTOPHER J. BUTLER 

4. ROBERT DAVID FARRAHER 

Execution Date: 1) 12/20/04, 2) 1/19/05, 3,4) 12/14/04, 


2. Name and address of receiving party(ies) 
Name: APRILIS, INC. 


Internal Address: 




Additional name(s) of conveying party(ies) attached? [ ] Yes [ X ] No 


Street Address: 5 CLOCK TOWER PLACE 


3. Nature of conveyance: 

[X ] Assignment [ ] Merger 

[ ] Security Agreement [ ] Change of Name 

[ ] Other 


SUITE 200 


City: MAYNARD 


State: MA 


Country: USA Zip: 01754 




Additional name(s) & address(es) attached? [ ] Yes [ X ] No 




4. Application number(s) or patent number(s): 
A. Patent Application No.(s) 
PCT/US2004/019713 

Additional numbers 


[ ] This document is being filed together with a new application. 
B. Patent No.(s) 

attached? [ ] Yes [ X ] No 


concerning document should be mailed: 
Name: TIMOTHY J. MEAGHER 


6. Total number of applications and patents involved: [ 1 ] 


7. Total Fee (37 C.F.R. 3.41) $ 40 


Internal Address: 


[ ] Enclosed 

[X ] Authorization to charge deposit account number 08-0380 
[ ] Previously submitted - Doc. ID No. [ ] 

rY1 Authorized to charge any deficiencies or credit any overpayment to 
L J deposit account number 08-03 80 

Do not attach a copy of this page if paying by deposit account and filing 
via facsimile. 

Attach a copy of this page if paying by deposit account and filing via mail. 


Hamilton, Brook, Smith & Reynolds, P.C. 


Street Address: 530 Virginia Road, P.O. Box 9133 
City: Concord State: MA ZIP: 01742-9133 


Phone No. 978-341-0036 Fax No. 978-341-0136 


Email Address: 




8. MARK B. SOLOMON 1 


IT? 0J~ t/t/es 


Name of Person Signing ( ^ Signature Date 
Total number of pages including cover sheet, attachments, and document: [ 5 ] 



Docket No. 3174.1012-006 PCT 



Joint 

ASSIGNMENT 

WHEREAS, we, Vincent Fedele, John S. Berg, Christopher J. Butler, and Robert 
David Farraher, have invented a certain improvement in Method and Apparatus for 
Processing Biometric Images described in an application for Patent, 

[ ] the specification of which is about to be filed in the United States Patent Office 
(use for utility (37 CFR § 1.53(b)) and design filings only); 

[ ] is about to be filed in the United States Patent Office as a Provisional Application; 

[ ] the specification of which is United States Application No. [ ], filed [ ]; 

[X ] the specification of which is a Patent Cooperation Treaty Application, 

International Application No. PCT/US2004/019713, filed June 21, 2004; 

[ ] which was patented under United States Patent No. [ ]. 

WHEREAS, Aprilis, Inc. (hereinafter "ASSIGNEE"), a corporation organized and 
existing under the laws of the State of Delaware, and having a usual place of business at 5 
Clock Tower Place, Suite 200, Maynard, MA 01754 desires to acquire an interest therein in 
accordance with agreements duly entered into with us; 

NOW, THEREFORE, to all whom it may concern be it known that for and in 
consideration of said agreements and of other good and valuable consideration, the receipt of 
which is hereby acknowledged, we have sold, assigned and transferred and by these presents do 
hereby sell, assign and transfer unto said ASSIGNEE, its successors, assigns and legal 
representatives, the entire right, title and interest in and throughout the United States of America, 
its territories and all foreign countries, in and to said invention as described in said application, 
together with the entire right, title and interest in and to said application and such Letters Patent 
as may issue on said invention; said invention, application and Letters Patent to be held and 
enjoyed by said ASSIGNEE for its own use and behalf and for its successors, assigns and legal 
representatives, to the full end of the term for which said Letters Patent may be granted as fully 
and entirely as the same would have been held by us had this assignment and sale not been made; 
we hereby convey all rights arising under or pursuant to any and all international agreements, 
treaties or laws relating to the protection of industrial property by filing any such applications for 
Letters Patent. We hereby acknowledge that this assignment, being of the entire right, title and 
interest in and to said invention, carries with it the right in ASSIGNEE to apply for and obtain 
from competent authorities in all countries of the world any and all Letters Patent by attorneys 
and agents of ASSIGNEE'S selection and the right to procure the grant of all such Letters Patent 
to ASSIGNEE for its own name as assignee of the entire right, title and interest therein; 

AND, we hereby further agree for ourselves and our executors and administrators to 
execute upon request any other lawful documents and likewise to perform any other lawful acts 
which may be deemed necessary to secure fully the aforesaid invention to said ASSIGNEE, its 
successors, assigns and legal representatives, but at its or their expense and charges, including the 
execution of applications for patents in foreign countries, and the execution of any future 
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DocketNo. 3174.1012-006 PCT 

applications including substitution, reissue, divisional or continuation applications, and 
preliminary or other statements and the giving of testimony in any interference or other 
proceeding in which said invention or any application or patent directed thereto may be involved; 

AND, we do hereby authorize and request each Patent Office and the Commissioner of 
Patents of the United States to issue such Letters Patent as shall be granted upon said invention to 
said ASSIGNEE, its successors, assigns, and legal representatives. 



IN TESTIMONY WHEREOF, we have hereunto set our hands and affixed our seals the 
date set forth below. 



Inventor's Signature: 

Vincent Fedele 

State/Commonwealth 

County of d^Wx 

On this q)Q day 0 f TW:^W^- 20 0 L * , before me, the undersigned 

notary public, personally appeared Vincent Fedele, 

| | personally known to me, or 

r~F\ proved to me through satisfactory evidence of identification, which was 

to be the person whose name was signed on the foregoing instrument in my presence, and 
acknowledged that he/she signed the foregoing instrument as his/her free act and deed. 



(SEAL) Cr AU^ 6A^£^ Notar y Public 

(\ \ l err, |Vj i „ qj^~ (P rint name > Colleen A. Beliveau 
L^\\e£.n wvu, NOTARY PUBLIC 

My Commission expires / / w commission ex0ires$eDf.2S 2009 
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Inventor's Signature: 

State/Commonwealth 
of _~j 



Docket No. 3174.1012-006 PCT 



jflL 



County of H jk^Ml^2^L 



On this day of Zlh/tit&ty 2 0 fir , before me, the undersigned 

notary public, personally appeared John S. Berg, 

| | personally known to me, or 

I — i proved to me through satisfactory_evidence of identification, which was 

[Or nic^irliujz^ LrojtJz 

to be the person whose name was signed on the foregoing instrument in my presence, and 
acknowledged that he/she signed the foregoing instrument as his/her free act and deed. 

(seal) / /aM&^dy Notary Public 




}/u/a ^&ft<*f (P^t name) 

My Commission expires 



Inventor's Signature: / J^u, <^W-i 



Christopher J. Butler 

State/Commonwealth / 

of ntft^L.suiist; 

County of (^Jjls^j^ 

On this I H day of "T^g/^W^^ , 20 ^ , before me, the undersigned 
notary public, personally appeared Christopher J. Butler, 

| | personally known to me, or 



roved to me through satisfactory evidence of identification, which was 

lu£ ±£LC^ 

to be the person whose name was signed on the foregoing instrument in my presence, and 
acknowledged that he/she signed the foregoing instrument as his/her free act and deed. 



(SEAL) /K)/^ Y/^JL Notary Public 

\^A^U ^^Ul1 ^ x name ) 

My Commission expires Up / '2—~l^Z>/<D 
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Inventor's Signature: 

~" * y Robert David Farraher 

State/Commonwealth 

of 

County of 

I day of \>£C&r.6£/l 20 0$ , before me, the undersigned 



notary public, personally appeared Robert David Farraher, 

| | personally known to n 



proved to me through satisfactory evidence of identification, which was 

AM i^pq- 

to be the person whose name was signed on the foregoing instrument in my presence, and 
acknowledged that he/she signed the foregoing instrument as his/her free act and deed. 

(SEAL) (^l^^ K^jlL ^ PUWiC 

\&A\&- 'T&^LCT- (print name) 
My Commission expires \\, I l2£>)Q 
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Conf i rmat i on Report-Memory Send 



Time : Feb-08-05 03:50pm 

Tel line 1 : 1-978-341-0242 

Tel line 2 : 1-978-341-0242 

Name : HBSR 



Job number 

Date 

To 

Document Pages 
Start time 
End time 
Pages sent 
Job number 
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Feb-08 03:48pm 

2*5908917033065995 
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*** SEND SUCCESSFUL *** 



HAMIL I ON, BROOK, SMITH &. REYNOLDS, V C 
530 Vir C mia Road. P.O. Box ©133 
Concord, MA 01742-9133 

Telephone: C97SJ 34 1-0036 

t yV<TSXJVl I I^I£- COVER SHEET 



Examiner: 
Date: 

Client Code: 
Facsimile No.: 
From: 
Subject: 



Facsimile: (978) 341-0136 



y 8, 200S 
3174 

703-306-S99S 
JVTaxlt B- Solomon 



3174.1012-006 
Applicant: Aprilis. Inc. 

Application No.: PCT/US2004/019713 
Filuig r>s»t«== June 21, 2004 



Number of pages i 



cip>t of facsimile: Yes 




HAMF ON, BROOK, SMITH & REYNOLr P.C. 
530 Virginia Road, P.O. Box 9133 
Concord, MA 01742-9133 



Telephone: (978) 341-0036 



Facsimile: (978) 341-0136 



FACSIMILE COVER SHEET 



Examiner: 



Assignment Division 

U.S. Patent and Trademark Office 



Date: February 8, 2005 

Client Code: 3174 

Facsimile No.: 703-306-5995 

From : Mark B. Solomon 

Subject: Paper: Assignment Recordation 

Docket No.: 3174.1012-006 

Applicant: Aprilis, Inc. 

Application No.: PCT/US2004/019713 

Filing Date: June 2 1 , 2004 

Number of pages including this cover sheet : 6 



Please confirm receipt of facsimile: Yes X No _ 



Privileged and Confidential - All information transmitted hereby is intended only for the use of the addressee(s) named above. If the 
reader of this message is not the intended recipient or the employee or agent responsible for delivering the message to the intended 
recipient s), please note that any distribution or copying of this communication is strictly prohibited. Anyone who received this 
communication in error is asked to notify us immediately by telephone and to destroy the original message or return it to us at the 
above address via first class mail. 
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This company uses RightFAX® fax 
server software from RightFAX, Inc. 
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From: Name: USPTO ASSIGNMENT DIVISION 

Fax Number: 

Voice Phone: 703-308-9723 



TO! Name: HAMILTON, BROOK, SMITH & REYNOLDS, P.C. 

Company: TIMOTHY J. MEAGHER 

Fax Number: 19783410242 
Voice Phone: 



Fax Notes: 

Pg# Description 

1 Cover Page 

2 276. TXT 

4 Document 1, Batch 418830 
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RECEIVED 

FEB 1 6 2005 
SMITH & REYNOLDS, P.c 



Date and time of transmission: Tuesday, February 15, 2005 6:26:22 PM 
Number of pages including this cover sheet: 04 
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Hamilton 



Brook 
Smith & 



Reynolds, PC. 



530 VIRGINIA ROAD 
P.O. BOX 9133 
CONCORD, MA 01742-9133 
TEL (978) 341-O036 
FAX (978) 341-0136 

www.hbsr.com 



March 6, 2006 



Munroe R Hamilton 



James M. Smith 



Robert T. Conway 
Rodney d. Johnson 
David J. Thibodeau, Jr. 
anne j. coujns 

Deirdree. Sanders 



David A. Waldman, Ph.D. 

Vice President, Research and Development 

Aprilis, Inc. 

5 Clock Tower Place 

Suite 200 

Maynard, Massachusetts 01754 

Re: U.S. National Stage Application No.: 10/561,647 

Title: "Method and Apparatus for Processing Biometric Images 
HBSR Docket No.: 3174.1012-014 



Michael J. Badzinski 



Dear David: 



Kristin a. Connarn 



Giovanna fessenden 



Enclosed is a Declaration for Patent Application for the referenced 
case and an Assignment form in which each inventor assigns rights to the 
invention to Aprilis, Inc. 

It is appropriate at this time to confirm that we have designated the 
correct inventor(s) in the attached Declaration and that the specification and 
claims meet the enablement and best mode requirements under U.S. patent 
law. 

To assist you in this determination, each inventor should note the 
following: 



Michael Gottseug 
Susan C. Kelly 

PAMELA A. TORPEY 



1 . Inventorship : Only persons who have 
made an inventive contribution to the 
conception of at least one of the claims in the 
patent application should be listed as an 
inventor. Any doubts on this matter should be 
resolved with me before proceeding further. 



2. Enablement : This requirement imposes 
an obligation on the inventor to describe the 
invention in sufficient detail to enable a person 
skilled in the art to make and use the invention 



David A. Waldman, Ph.D. 
March 6, 2006 
Page 2 



which is the subject of the patent application without undue 
experimentation. 

3. Best Mode : The "best mode" requirement goes further than 
the "enablement" requirement by obligating the inventor to include 
in the patent application a description of the best mode 
contemplated by the inventor of practicing the invention. 



An inventor's failure to comply with the "best mode" or "enablement" requirements may 
invalidate a patent. In addition, intentional failure to correctly name inventors could result in an 
invalid or unenforceable patent. If you have any questions or concerns, please call me. 

Please note that the Declaration requires that each inventor carefully read the patent 
application in its entirety before signing it. If the application is acceptable, please have each 
inventor sign the Declaration in BLUE ink. Each inventor should also check to be sure that the 
typed name, address and citizenship on the Declaration are correct. If they are not, the inventor 
should indicate the correct information in BLUE ink, and initial and date each correction in the 
margin. 

If you have any questions or concerns, or feel any changes need to be made in the 
Declaration, please call me. 

The Assignment should be signed in BLUE ink by each inventor in the presence of a 
Notary Public. 

Please return all pages of these documents as soon as possible for filing in the U.S. Patent 
and Trademark Office. 



Very truly yours, 



HAMILTON, BROOK, SMITH & REYNOLDS, P.C. 




Alexander Akhiezer 



AA/saj 
Enclosures 



cc: 



Timothy J. Meagher, Esq. 
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DOCKET NO. 3174.1012-014 



IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 



Declaration for Patent Application 



[] Supplemental (37 C.F.R. §1.67) 



As a named inventor, I hereby declare that: 

My residence, mailing address and citizenship are as stated next to my name; 

I believe I am the original, first and sole inventor (if only one name is listed) or an original, first and joint 
inventor (if plural names are listed in the signatory page(s) commencing at page 2 hereof) of the subject matter which 
is claimed and for which a patent is sought on the invention entitled 

METHOD AND APPARATUS FOR PROCESSING BIOMETRIC IMAGES 



the specification of which (check one) 
[ ] is attached hereto. 

[] was filed on [ ] as United States Application Number [ ]. 

[X ] was filed on June 2 1 , 2004 as PCT International Application No. PCT/US2004/0 1 97 1 3 and assigned United 
States Application No. 10/561,647. 

[ ] and was amended on [ ] (if applicable). 

I hereby state that I have reviewed and understand the contents of the above-identified specification, 
including the claims, as amended by any amendment referred to above. 

I acknowledge the duty to disclose information which is material to patentability as defined in 37 C.F.R. 
§1.56, including for continuation-in-part applications, material information which became available between the 
filing date of the prior application and the national or PCT international filing date of the continuation-in-part 
application. 

I hereby expressly authorize the filing of an International Patent Application under the Patent Cooperation 
Treaty which corresponds to and claims the priority of the above-identified application. 

I hereby claim foreign priority benefits under 35 U.S.C. 1 19 or 365 of any foreign application(s) for patent 
or inventor's certificate, or of any PCT international application which designated at least one country other than the 
United States of America, listed below and have also identified below, by checking the box, any foreign application 
for patent or inventor's certificate, or of any PCT international application having a filing date before that of the 
application on which priority is claimed: 



Prior Foreign Appiication(s) 



Priority 

Not 
Claimed 



Certified 
Copy Filed? 
YES NO 



[ ] 



[ ] 



[ ] 



(Number) 



(Country) 



(Day/Month/Year filed) 



[ ] 



[ ] 



[ ] 



(Number) 



(Country) 



(Day/Month/Year filed) 



[ ] [ 1 [ 1 

(Number) (Country) (Day/Month/Year filed) 
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I hereby declare that all statements made herein of my own knowledge are true and that all statements made 
on information and belief are believed to be true; and further that these statements were made with the knowledge 
that willful false statements and the like so made are punishable by fine or imprisonment, or both, under Section 
1001 of Title 18 of the United States Code and that such willful false statements may jeopardize the validity of the 
application or any patent issued thereon. 



Full name of sole 
or first inventor 


Vincent Fedele 






Inventor's Signature 








Residence: City 


State 


Country 


Citizenship 


| Harvard 


I MA 


! USA 


i USA | 


Mailing Address (Bus 
Street or P.O. Box 


iness or Residential) 
P.O. Box 061 






City 


State 


ZIP 


Country 


| Harvard 


MA 


01451 


1 USA | 




Full name of second 
joint inventor, if any 


John S. Berg 






Inventor's Signature 








Residence: City 


State 


Country 


Citizenship 


I Franklin 


| MA 


USA 


i USA j 


Mailing Address (Business or Residential) 
Street or P.O. Box 58 Jefferson Road 


City 


State 


ZIP 


Country 


Franklin 


I MA 


| 02038-3360 


! usa ! 




Full name of third 
j oint inventor, if any 


Christopher J. Butler 






Inventor's Signature 








Residence: City 


State 


Country 


Citizenship 


I Arlington 


1 MA 


I USA 


i USA i 


Mailing Address (Business or Residential) 
Street or P.O. Box 269 Broadway, Suite 9 


City 


State 


ZIP 


Country 


j Arlington 


1 MA 


I 02474 


! USA 




Full name of fourth 
joint inventor, if any 


Robert David Farraher 






Inventor's Signature 








Residence: City 


State 


Country 


Citizenship 


! Somerville 


! MA 


[ USA 


1 USA ! 


Mailing Address (Bus 
Street or P.O. Box 


iness or Residential) 
24 Main Street, #2 






City 


State 


ZIP 


Country 


| Somerville 


! MA 


i 02145 


! USA ! 
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Joint 

ASSIGNMENT 

WHEREAS, we, Vincent Fedele, John S. Berg, Christopher J. Butler and Robert 
David Farraher, have invented a certain improvement in METHOD AND APPARATUS FOR 
PROCESSING BIOMETRIC IMAGES described in an application for Patent, 

[ ] the specification of which is about to be filed in the United States Patent Office 
(use for utility (37 CFR § 1.53(b)) and design filings only); 

[ ] is about to be filed in the United States Patent Office as a Provisional Application; 

[ ] the specification of which is United States Application No. [ ], filed [ ]; 

[ X ] the specification of which is a Patent Cooperation Treaty Application, 

International Application No. PCI7US2004/019713, filed June 21, 2004, which 
designates the United States of America and assigned United States Application 
No. 10/561,647; 

[] which was patented under United States Patent No. [ ]. 

WHEREAS, Aprilis, Inc. (hereinafter "ASSIGNEE"), a corporation organized and 
existing under the laws of the State of Delaware, and having a usual place of business at 
5 Clock Tower Place, Suite 200, Maynard, Massachusetts 01754 desires to acquire an interest 
therein in accordance with agreements duly entered into with us; 

NOW, THEREFORE, to all whom it may concern be it known that for and in 
consideration of said agreements and of other good and valuable consideration, the receipt of 
which is hereby acknowledged, we have sold, assigned and transferred and by these presents do 
hereby sell, assign and transfer unto said ASSIGNEE, its successors, assigns and legal 
representatives, the entire right, title and interest in and throughout the United States of America, 
its territories and all foreign countries, in and to said invention as described in said application, 
together with the entire right, title and interest in and to said application and such Letters Patent 
as may issue on said invention; said invention, application and Letters Patent to be held and 
enjoyed by said ASSIGNEE for its own use and behalf and for its successors, assigns and legal 
representatives, to the full end of the term for which said Letters Patent may be granted as fully 
and entirely as the same would have been held by us had this assignment and sale not been made; 
we hereby convey all rights arising under or pursuant to any and all international agreements, 
treaties or laws relating to the protection of industrial property by filing any such applications for 
Letters Patent. We hereby acknowledge that this assignment, being of the entire right, title and 
interest in and to said invention, carries with it the right in ASSIGNEE to apply for and obtain 
from competent authorities in all countries of the world any and all Letters Patent by attorneys 
and agents of ASSIGNEE'S selection and the right to procure the grant of all such Letters Patent 
to ASSIGNEE for its own name as assignee of the entire right, title and interest therein; I hereby 
expressly authorize the filing of an International Patent Application under the Patent Cooperation 
Treaty which corresponds to and claims the priority of the above-identified application; 
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AND, we hereby further agree for ourselves and our executors and administrators to 
execute upon request any other lawful documents and likewise to perform any other lawful acts 
which may be deemed necessary to secure fully the aforesaid invention to said ASSIGNEE, its 
successors, assigns and legal representatives, but at its or their expense and charges, including the 
execution of applications for patents in foreign countries, and the execution of any future 
applications including substitution, reissue, divisional or continuation applications, and 
preliminary or other statements and the giving of testimony in any interference or other 
proceeding in which said invention or any application or patent directed thereto may be involved; 

AND, we do hereby authorize and request each Patent Office and the Commissioner of 
Patents of the United States to issue such Letters Patent as shall be granted upon said invention to 
said ASSIGNEE, its successors, assigns, and legal representatives. 



IN TESTIMONY WHEREOF, we have hereunto set our hands and affixed our seals the 
date set forth below. 



Inventor's Signature: 

Vincent Fedele 

State/Commonwealth 

of 



County of 



On this day of , 20 , before me, the undersigned 

notary public, personally appeared Vincent Fedele, 

| | personally known to me, or 

| 1 proved to me through satisfactory evidence of identification, which was 

to be the person whose name was signed on the foregoing instrument in my presence, and 
acknowledged that he/she signed the foregoing instrument as his/her free act and deed. 

(SEAL) Notary Public 

(print name) 

My Commission expires / / 
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Inventor's Signature: 

John S. Berg 

State/Commonwealth 

of 



County of 



On this day of , 20 , before me, the undersigned 

notary public, personally appeared John S. Berg, 

personally known to me, or 

| — | proved to me through satisfactory evidence of identification, which was 

to be the person whose name was signed on the foregoing instrument in my presence, and 
acknowledged that he/she signed the foregoing instrument as his/her free act and deed. 

(SEAL) Notary Public 

(print name) 

My Commission expires / / 

Inventor's Signature: 

Christopher J. Butler 

State/Commonwealth 

of 

County of 



On this day of , 20 , before me, the undersigned 

notary public, personally appeared Christopher J. Butler, 

personally known to me, or 

| 1 proved to me through satisfactory evidence of identification, which was 

to be the person whose name was signed on the foregoing instrument in my presence, and 
acknowledged that he/she signed the foregoing instrument as his/her free act and deed. 

(SEAL) Notary Public 

(print name) 

My Commission expires / / 
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Inventor's Signature: 

Robert David Farraher 

State/Commonwealth 

of 



County of 



On this day of , 20 , before me, the undersigned 

notary public, personally appeared Robert David Farraher, 

| | personally known to me, or 

| — | proved to me through satisfactory evidence of identification, which was 

to be the person whose name was signed on the foregoing instrument in my presence, and 
acknowledged that he/she signed the foregoing instrument as his/her free act and deed. 

(SEAL) Notary Public 

(print name) 

My Commission expires / / 
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Weaver yr) 
Austin IT 
Villeneuve 
Sampson 

Intellectual Property Law 

June 4, 2008 

Via Express Mail 

Vincent A. Fedele 

P.O. Box 061 
Harvard, MA 01451 



Re: U.S. Patent Application Entitled: "Method and Apparatus for Processing Biometric 
Images" 

Inventors: Fedele, et al. Application No: 10/561,647 

Filing Date: December 20, 2005 
Our File: IDTXP052B 



Dear Mr. Fedele: 

Enclosed is a copy of the above-identified patent application as filed with the U.S. Patent 
and Trademark Office. You may recall that the application was filed without having you sign 
the necessary documents (declaration and assignment forms) which need to be filed with the U.S. 
Patent and Trademark Office by June 20, 2008. 



Accordingly, we are enclosing two documents: (i) a Declaration and Pow, r of Attorney 
and (ii) an Assignment of rights to Aprilis, Inc. Before executing these documents, please ensure 
that you have reviewed and understand the contents of the enclosed patent application. Then, 
read, sign and date each of the enclosed documents adjacent to where your name appears, and 
return them to our office so that we may file them with the U.S. Patent and Trademark Office. 

Finally, we would again like to remind you of our duty to disclose the most pertinent 
prior art of which you are aware to the Patent and Trademark Office. If you can think of any 
pertinent references or patents, or any similar existing technology, please let us know. The duty 
to disclose prior art continues until the patent actually issues. As such, if you become aware of 
other prior art in the future, please let us know. 

Sincerely, 

Weaver Austin Villeneuve & Sampson LLP 
Denise S. Bergin^J 

Cc: Tom McPherson 
Corporate Counsel 
Identix Incorporated 
5600 Rowland Road 
Minnetonka, MN 55343 

Enclosures 
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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 



Declaration for Patent Application 



[ ] Supplemental (37 C.F.R. §1.67) 



As a named inventor, I hereby declare that: 

My residence, mailing address and citizenship are as stated next to my name; 

I believe I am the original, first and sole inventor (if only one name is listed) or an original, first and joint 
inventor (if plural names are listed in the signatory page(s) commencing at page 2 hereof) of the subject matter which 
is claimed and for which a patent is sought on the invention entitled 

METHOD AND APPARATUS FOR PROCESSING BIOMETRJC IMAGES 



the specification of which (check one) 
[ ] is attached hereto. 

[] was filed on [ ] as United States Application Number [ ]. 

[X ] was filed on June 21, 2004 as PCT International Application No. PCT/US2004/019713 and assigned United 
States Application No. 10/561,647. 

[ ] and was amended on [ ] (if applicable). 

I hereby state that I have reviewed and understand the contents of the above-identified specification, 
including the claims, as amended by any amendment referred to above. 

I acknowledge the duty to disclose information which is material to patentability as defined in 37 C.F.R. 
§1.56, including for continuation-in-part applications, material information which became available between the 
filing date of the prior application and the national or PCT international filing date of the continuation-in-part 
application. 

I hereby expressly authorize the filing of an International Patent Application under the Patent Cooperation 
Treaty which corresponds to and claims the priority of the above-identified application. 

I hereby claim foreign priority benefits under 35 U.S.C. 1 19 or 365 of any foreign application(s) for patent 
or inventor's certificate, or of any PCT international application which designated at least one country other than the 
United States of America, listed below and have also identified below, by checking the box, any foreign application 
for patent or inventor's certificate, or of any PCT international application having a filing date before that of the 
application on which priority is claimed: 



Prior Foreign Application(s) 



Priority 

Not 
Claimed 



Certified 
Copy Filed? 
YES NO 



[ ] 



[ ] 



[ 1 



(Number) 



(Country) 



(Day/Month/Year filed) 



[ ] 



[ ] 



[ ] 



(Number) 



(Country) 



(Day/Month/Year filed) 



[ ] [ ] [ ] 

(Number) (Country) (Day/Month/Year filed) 
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I hereby declare that all statements made herein of my own knowledge are true and that all statements made 
on information and belief are believed to be true; and further that these statements were made with the knowledge 
that willful false statements and the like so made are punishable by fine or imprisonment, or both, under Section 
1001 of Title 18 of the United States Code and that such willful false statements may jeopardize the validity of the 
application or any patent issued thereon. 



Full name of sole 
or first inventor 


Vincent Fedele 






Inventor's Signature 


Residence: City 


State 


Country 


Citizenship 


j Harvard 


! MA 


| USA 


I USA 


Mailing Address (Busi 
Street or P.O. Box 


ness or Residential) 
P.O. Box 061 






City 


State 


ZIP 


Country 


I Harvard 


! MA 


i 01451 


i USA 




Full name of second 
joint inventor, if any 


John S. Berg 






Inventor's Signature 


Residence: City 


State 


Country 


Citizenship 


1 Franklin 


I MA 


I USA 


1 USA 


Mailing Address (Bust 
Street or P.O. Box 


ness or Residential) 
58 Jefferson Road 






City 


State 


ZIP 


Country 


i Franklin 


i MA 


j 02038-3360 


| USA 




Full name of third 
joint inventor, if any 


Christopher J. Butler 






Inventor's Signature 


Residence: City 


State 


Country 


Citizenship 


I Arlington 


I MA 


! USA 


f USA 


Mailing Address (Business or Residential) 
Street or P.O. Box 269 Broadway, Suite 9 


City 


State 


ZIP 


Country 


Arlington 


1 MA 


1 02474 


| USA ! 




Full name of fourth 
joint inventor, if any 


Robert David Farraher • 






Inventor's Signature 


Residence: City 


State 


Country 


Citizenship 


| Somerville 


! MA 


i USA 


| USA 


Mailing Address (Bush 
Street or P.O. Box 


ness or Residential) 
24 Main Street, #2 






City 


State 


ZIP 


Country 


f Somerville 


I MA 


| 02145 


| USA 



^FDesktop\::ODMAMHODl*VHBSR0S;iManage;606673;l 



Docket No. 3174.1012-014 



Joint 

ASSIGNMENT 

WHEREAS, we, Vincent Fedele, John S. Berg, Christopher J. Butler and Robert 
David Farraher, have invented a certain improvement in METHOD AND APPARATUS FOR 
PROCESSING BIOMETRIC IMAGES described in an application for Patent, 

[ ] the specification of which is about to be filed in the United States Patent Office 
(use for utility (37 CFR § 1.53(b)) and design filings only); 

[ ] is about to be filed in the United States Patent Office as a Provisional Application; 

[ ] the specification of which is United States Application No. [ ], filed [ ]; 

[ X ] the specification of which is a Patent Cooperation Treaty Application, 

International Application No. PCT/US2004/019713, filed June 21, 2004, which 
designates the United States of America and assigned United States Application 
No. 10/561,647; 

[ ] which was patented under United States Patent No. [ ] . 

WHEREAS, Aprilis, Inc. (hereinafter "ASSIGNEE"), a corporation organized and 
existing under the laws of the State of Delaware, and having a usual place of business at 
5 Clock Tower Place, Suite 200, Maynard, Massachusetts 01754 desires to acquire an interest 
therein in accordance with agreements duly entered into with us; 

NOW, THEREFORE, to all whom it may concern be it known that for and in 
consideration of said agreements and of other good and valuable consideration, the receipt of 
which is hereby acknowledged, we have sold, assigned and transferred and by these presents do 
hereby sell, assign and transfer unto said ASSIGNEE, its successors, assigns and legal 
representatives, the entire right, title and interest in and throughout the United States of America, 
its territories and all foreign countries, in and to said invention as described in said application, 
together with the entire right, title and interest in and to said application and such Letters Patent 
as may issue on said invention; said invention, application and Letters Patent to be held and 
enjoyed by said ASSIGNEE for its own use and behalf and for its successors, assigns and legal 
representatives, to the full end of the term for which said Letters Patent may be granted as fully 
and entirely as the same would have been held by us had this assignment and sale not been made; 
we hereby convey all rights arising under or pursuant to any and all international agreements, 
treaties or laws relating to the protection of industrial property by filing any such applications for 
Letters Patent. We hereby acknowledge that this assignment, being of the entire right, title and 
interest in and to said invention, carries with it the right in ASSIGNEE to apply for and obtain 
from competent authorities in all countries of the world any and all Letters Patent by attorneys 
and agents of ASSIGNEE'S selection and the right to procure the grant of all such Letters Patent 
to ASSIGNEE for its own name as assignee of the entire right, title and interest therein; I hereby 
expressly authorize the filing of an International Patent Application under the Patent Cooperation 
Treaty which corresponds to and claims the priority of the above-identified application; 
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AND, we hereby further agree for ourselves and our executors and administrators to 
execute upon request any other lawful documents and likewise to perform any other lawful acts 
which may be deemed necessary to secure fully the aforesaid invention to said ASSIGNEE, its 
successors, assigns and legal representatives, but at its or their expense and charges, including the 
execution of applications for patents in foreign countries, and the execution of any future 
applications including substitution, reissue, divisional or continuation applications, and 
preliminary or other statements and the giving of testimony in any interference or other 
proceeding in which said invention or any application or patent directed thereto may be involved; 

AND, we do hereby authorize and request each Patent Office and the Commissioner of 
Patents of the United States to issue such Letters Patent as shall be granted upon said invention to 
said ASSIGNEE, its successors, assigns, and legal representatives. 



IN TESTIMONY WHEREOF, we have hereunto set our hands and affixed our seals the 
date set forth below. 



Inventor's Signature: 

Vincent Fedele 

State/Commonwealth 

of 



County of 



On this day of , 20 , before me, the undersigned 

notary public, personally appeared Vincent Fedele, 

| | personally known to me, or 

| — | proved to me through satisfactory evidence of identification, which was 

to be the person whose name was signed on the foregoing instrument in my presence, and 
acknowledged that he/she signed the foregoing instrument as his/her free act and deed. 

(SEAL) Notary Public 

(print name) 

My Commission expires / / 



EXHIBIT C 
(part 2) 



(12) INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(19) World Intellectual Property 
Organization 

International Bureau 




(43) International Publication Date (10) International Publication Number 

6 January 2005 (06.01.2005) PCT WO 2005/001752 A2 



(51) International Patent Classification 7 : 



(22) International Filing Date: 21 June 2004 (21.06.2004) 



(25) Filing Language: 

(26) Publication Language: 



English 
English 



(30) Priority Data: 

60/480,008 21 June 2003 (21.06.2003) US 

60/519,792 13 November 2003 (13.1 1.2003) US 

j 60/523,068 18 November 2003 (18.11.2003) US 

; (71) Applicant (for all designated States except US): APWLIS, 

| INC. [US/US]; 5 Clock Tower Place, Suite 200, Maynard, 

j MA 01754 (US). 

= (72) Inventors; and 

; (75) Inventors/Applicants (for US only): FEDELE, Vincent 

: [US/US]; Post Office Box 061, Harvard, MA 01451 (US). 

| BERG, John, S. [US/US]; 58 Jefferson Road, Franklin, 

| MA 02038 (US). BUTLER, Christopher, J. [US/US]; 

j Suite 9, 269 Broadway, Arlington, MA 02474 (US). 

! FARRAHER, Robert, David [US/US]; 24 Main Street 

= #2, Somerville, MA 02145 (US). 



(74) Agents: MEAGHER, Timothy, J. et al.; Hamilton, 
Brook, Smith & Reynolds, P.C., 530 Virginia Road, P.O. 
Box 9133, Concord, MA 01742-9133 (US). 

(81) Designated Stales {unless otherwise indicalal. for every 
kind of national protection available): AE, AG, AL, AM, 
AT, AU, AZ, BA, BB, BG, BR, BW, BY, BZ, CA, CH, CN, 
CO, CR, CU, CZ, DE, DK, DM, DZ, EC, EE, EG, ES, FI, 
GB, GD, GE. Gil, GM, IIR. HU, ID, IL, IN, IS, JP, KE, 
KG, KP. KR, KZ, LC, LK, LR, LS, LT, LU, LV, MA, MD, 
MG, MK, MN, MW, MX, MZ, NA, NI, NO, NZ, OM, PG, 
PH, PL, PT, RO, RU, SC, SD, SE, SG, SK, SL, SY, TJ, TM, 
TN, TR, IT, TZ, UA, UG, US, UZ, VC, VN, YU, ZA, ZM, 
ZW. 

(84) Designated States (unless otherwise indicated, for every 
kind of regional protection available): ARIPO (BW, GH, 
GM, KE, LS, MW, MZ, NA, SD, SL, SZ, TZ, UG, ZM, 
ZW), Eurasian (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), 
European (AT, BE, BG, CI I, CY. CZ. DE, DK, EE, ES, FI, 
FR, GB, GR, HU, IE, IT, LU, MC, NL, PL, PT, RO, SE, SI, 
SK, TR), OAPI (BF, BJ, CF, CG, CI, CM, GA, GN, GQ, 
GW, ML, MR, NE, SN, TD, TG). 

Published: 

— without international search report and to be republished 
upon receipt of that report 

For two '-lei i ei codt", and other ahhrc\ unions, refer to the "Guid- 
ance Notes on Codes and Abbreviations" appearing at the begin- 
ning of each regular issue of the PCT Gazette. 



" (54) Title: METHOD AND APPARATUS FOR PROCESSING BIOMETRIC IMAGES 




o 

O (57) Abstract: A method and apparatus for applying gradient edge detection to detect features in a biometric, such as a fingerprint, 
^ based on data representing an image of at least a portion of the biometric. The image is modeled as a function of the features. Data 
^ representing an image of the biometric is acquired, and features of the biometric arc modeled for at least two resolutions. The method 
and apparatus improves analysis of both high-resolution images of biometrics of friction ridge containing skin that include resolved 
j^- pores and lower resolution images of biometrics without resolved pores. 



WO 2005/001752 



PCT7US2004/019713 



METHOD AND APPARATUS FOR PROCESSING BIOMETRIC IMAGES 

RELATED APPLICATION(S) 

This application claims the benefit of U.S. Provisional Application No. 
60/480,008, filed on June 21, 2003, U.S. Provisional Application No. 60/519,792, 
5 filed on November 13, 2003, and U.S. Provisional Application No. 60/523,068, filed 
on November 18, 2003. This application is related to PCT Application entitled 
"Acquisition of High Resolution Biometric Images," filed concurrently herewith on 
June 21, 2004 under Attorney Docket No. 3174.1012-004. The entire teachings of 
the above applications are incorporated herein by reference. 

1 0 BACKGROUND OF THE INVENTION 

Growing concerns regarding domestic security have created a critical need to 
positively identify individuals as legitimate holders of credit cards, driver's licenses, 
passports and other forms of identification. The ideal identification process is 
reliable, fast, and relatively inexpensive. It should be based on modern, high-speed, 

1 5 electronic devices that can be networked to enable fast and effective sharing of 

information. It should also be compact, portable, and robust for convenient use in a 
variety of environments, including airport security stations, customs and border 
crossings, police vehicles, point of sale applications, credit card and ATM 
applications, home and office electronic transactions, and entrance control sites to 

20 secure buildings. 

A well established method for identification or authentication is to compare 
biometric characteristics of an individual with a previously obtained authentic 
biometric of the individual. Possible biometric characteristics include ear shape and 
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structure, facial characteristics, facial or hand thermograms, iris and retina structure, 
handwriting, and friction ridge patterns of skin such as fingerprints, palm prints, foot 
prints, and toe prints. A particularly useful biometric system uses fingerprints for 
individual authentication and identification. (Maltoni, Maio, Jain, and Prabhakar, 
5 "Handbook of Fingerprint Recognition", Springer, 2003, chapter 1, and David R. 
Ashbaugh, "Quantitative-Qualitative Friction Ridge Analysis", CRC Press, 1999). 

Fingerprints have traditionally been collected by rolling an inked finger on a 
white paper card. Since this traditional process clearly fails to meet the criteria 
listed above, numerous attempts have been made to develop an electronically 

10 imaged fingerprint method to address new security demands. These modern 

methods typically use, as a key component, a solid-state device such as a capacitive 
or optical sensor to capture the fingerprint image in a digital format. By using a 
solid-state imager as part of a fingerprint identification apparatus, a fingerprint can 
be collected conveniently and rapidly during a security check, for example, and 

15 subsequently correlated, in near real-time, to previously trained digital fingerprints 
in an electronic database. The database can reside on a computer at the security 
check point, on a secure but portable or removable storage device, on a remotely 
networked server, or as a biometric key embedded into a smartcard, passport, 
license, birth certificate, or other form of identification. 

20 The topological features of a typical finger comprise a pattern of ridges 

separated by valleys, and a series of pores located along the ridges. The ridges are 
typically 100 to 300 urn wide and can extend in a number of different swirl-like 
patterns for several mm to one or more cm. The ridges are separated by valleys 
with a typical ridge-valley period of approximately 250-500 um. Pores, roughly 

25 circular in cross section, range in diameter from about 40 um to 200 um, and are 
aligned along the ridges. The patterns of both ridges/valleys and pores are believed 
to be unique to each fingerprint. No currently available commercial fingerprint 
acquisition technique is able to resolve pores and ridge deviation details to a degree 
necessary to use this vastly larger amount of information as a biometric key. 

30 Accordingly, present-day automatic fingerprint identification procedures use only 
portions of ridge and valley patterns, called minutiae, such as ridge ending-points, 
deltoids, bifurcations, crossover points, and islands, which are found in almost every 
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fingerprint (Maltoni, Maio, Jain, and Prabhakar, "Handbook of Fingerprint 
Recognition", Springer, 2003, chapter 3). Extraction and comparison of minutiae is 
the basis of most current automatic fingerprint analysis systems. 

There are several important limitations with minutiae-based methods of 
5 automatic fingerprint analysis. In order to collect enough minutiae for reliable 

analysis a relatively large area, at least 0.50 x 0.50 inches, good quality, fingerprint, 
or latent image of a fingerprint must be available. Large prints are often collected 
by rolling an inked finger on a white card, and subsequently scanning the inked 
image into an electronic database. This manual procedure is an awkward and time 

10 consuming process that requires the assistance of a trained technician. Automated 
methods for collecting large fingerprints usually require mechanically complicated 
and expensive acquisition devices. Large area fingerprints suffer from distortions 
produced by elastic deformations of the skin so that the geometrical arrangements 
between minutiae points vary from image to image of the same finger, sometimes 

15 significantly. In addition, forensic applications can involve small, poor quality, 
latent prints that contain relatively few resolved minutiae so that reliable analysis 
based on a limited number of minutiae points is quite difficult. 

Minutiae comparison ignores a significant amount of structural information 
that may be used to enhance fingerprint analysis. Since the typical fingerprint 

20 contains between 7 to 10 times as many pores as minutiae, techniques that include 
both pores and minutiae should greatly improve matching compared to techniques 
that use only minutiae. This highly detailed information is referred to in the industry 
as "level three detail," and is the basis of most forensic level analysis of latent 
images left at a crime scene, where the latent does not contain enough minutiae to 

25 make an accurate identification. Stosz and Alyea (J. D. Stosz, L. A. Alyea, 

"Automated system for fingerprint authentication using pores and ridge structures", 
Proc. SPIE, vol 2277, 210-223, 1994) have confirmed this expectation by showing 
that the use of pores combined with minutiae improves the accuracy of fingerprint 
matching and allows successful analysis of relatively small prints. Their image 

30 sensor used a common prism-based configuration, a high-resolution Charge Coupled 
Device (CCD) video camera, and a macro lens to provide the resolution needed to 
image pores. After acquisition, the gray-scale images are converted to a binary 
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format and then processed further to produce a skeleton image from which minutiae 
and pores are identified. Fingerprints are compared by independent correlations 
between pores and minutiae extracted from the various images. 

SUMMARY OF THE INVENTION 
5 There is a need for a procedure that improves an analysis of both high- 

resolution images of biometrics (e.g., fingerprints that include resolved pores) and 
lower resolution images of biometrics (e.g., fingerprints without resolved pores). 
The principles of the present invention fulfill this need by using identifying 
information in a biometric, which, in the case of a fingerprint, can include 

10 fingerprint ridge shapes or profiles in addition to usual ridge contours and the 
position, shape, and sizes of pores. Images to be analyzed may include biometric 
images, such as fingerprints, (i) from an apparatus custom-designed to capture such 
images either in real-time or non-real-time, or (ii) from other apparatus (e.g., 
computer scanner) that scans crime scene latent images, as well as existing criminal 

1 5 arrest or civil-applicant background check records. 

Accordingly, one embodiment of the principles of the present invention 
includes a method and apparatus for processing an image of a biometric, which, for 
purposes of illustration only, is described in detail herein in reference to acquiring 
and processing an image of a fingerprint. The method and apparatus, referred to 

20 generally here as "system," may apply a gradient edge detection process to detect 
features in a biometric based on data representing an image of at least a portion of 
the fingerprint. The system models the image as a function of the fingerprint 
features, which may include level three features. The models may be referred to 
herein as "trained" models. 

25 The system may construct a model for at least two resolutions: a low 

resolution "outline" model and a high resolution "details" model. The outline model 
may generally show an edge topology of ridge features; the details model generally 
shows edge topology and specific ridge deviations and locations and sizes of pores. 
The system may also construct a model for a third resolution, a "fine details" model. 

30 The fine details model is used for precisely defining and locating particular 

biometric features more accurately than at the low or high resolutions, such as pores 
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in a fingerprint image. It is this third resolution model that is used, for example, for 
pore matching in authentication and identification processes in a fingerprint 
application. 

In constructing the model of a fingerprint, the system may identify, outline, 
5 and extract ridge deviation detail and pore features. The ridge deviation detail may 
include ridge contours, including scars, and the pore features may include position, 
shape, and sizes of pores. 

Biometrics for which the system is adapted to acquire, model, preprocess, 
and process may include: ear shape and structure, facial or hand thermograms, iris 

10 or retina structure, handwriting, and friction ridge patterns of skin such as 
fingerprints, palm prints, foot prints, and toe prints. 

The system may construct models at various resolution levels through a 
process of binning the original image data. In this process, the image data is divided 
into equal-sized, sub arrays of pixels. Each pixel sub array is subsequently 

1 5 represented in the model by a single pixel whose value is equal to the average pixel 
value in the corresponding sub array. The sizes of the sub arrays can be adjusted by 
a user of the software to any appropriate value; typical examples follow for a CMOS 
or CCD sensor array, described below in reference to Fig. 4, and having an array of 
1024 x 1280 6 um square pixels. "Outline" resolution models may be constructed 

20 with a sub-arrays having a relatively large numbers of pixels, for example 10 to 15, 
"details" resolution models may be constructed with sub arrays having a relatively 
small number of pixels, for example 5 to 10, and "fine details" models may be 
constructed with sub-arrays having even fewer pixels, for example 2 to 5. The fine 
details model may be used to locate and define particular biometric features more 

25 accurately than at the low or high resolution; for example, in the case of fingerprints, 
pores on ridges may be located and defined in the fine details model. 

The gradient determined by the gradient edge detection process may be 
estimated for each pixel of the model after applying a noise filter to the image. The 
detection process may use a finite-differences process. The detection process may 

30 also include a series of steps, such as the following: (a) after calculating the 

gradients, identifying and marking an image point as an edge point having a locally 
maximal gradient in the direction of the gradient that exceeds a threshold; (b) 
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identifying neighboring edge points by finding nearest pixels to the original edge 
point that lie in a direction that is approximately perpendicular to the gradient 
direction that passes through the first edge point; (c) for the nearest pixels, 
determining gradient values and, for the pixel with a gradient that is maximal along 
5 its gradient direction and has a value that exceeds a threshold, assigning the pixel to 
be the next edge point; (d) continuing either until the edge is terminated or the edge 
closes with itself to form a continuous curve; (e) terminating the process at the 
previously determined edge point if the gradient of a candidate edge point is less 
than the threshold; and (f) repeating the process until all potential edge points have 

10 been considered. 

The system may automatically distinguish biometric features from noise. In 
one embodiment, the noise is defined as features that have less than a minimum 
width or extend less than a minimum distance. In addition to automatically 
distinguishing the biometric features from noise, the system may also support 

1 5 manual editing of features and/or manual selection of features that must be present 
for a successful match. 

The system may model multiple regions of a single image of the portion of 
the biometric. For example, the models may be models of five regions of the 
biometric, such as four quadrants of the biometric with small overlaps in each 

20 quadrant, and may also include a center portion that overlaps portions of each of the 
four quadrants. The system may allow a user to add, extend, or delete features and 
may allow a user to identify specific or unique features that must be present for a 
match. The system may also allow a user to adjust the size or position of the 
model(s) relative to the biometric. User interaction may be performed through a 

25 Graphical User Interface (GUI) supported by the system. 

A useful aspect of this technique for edge detection is an ability to detect 
edges even if a portion or all of the image is significantly over and/or underexposed, 
as a few levels of gray difference are sufficient to determine a location of an edge. 
This allows for highly accurate matching even if the incoming image or portion of 

30 the image for comparison is not properly exposed, which allows for minimal or no 
exposure correction. 
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The image may be a previously stored image, and the system may normalize 
the scale or size of the previously stored image so that the scale is similar to that of 
the trained model(s). This scaling calibration also allows highly accurate 
measurements to be taken for forensic purposes. Typical accuracy of such 
5 measurements may be better than lOum. 

In the case where the biometrics are fingerprints, the fingerprint features may 
include ridge structure with ridge deviation detail. Further, for fingerprints and 
other biometrics, the system may display the image to a user with an overlay of 
indications of the biometric features on the image or filtered biometric features 
10 according to a selectable criteria. Also, the system may automatically rotate the 
image to a specified orientation for displaying to a user and can rotate and scale the 
image while performing a match. In one embodiment, the image is a gray-scale 
image, and the gradient edge detection process is a gray-scale gradient edge 
detection process. 

1 5 The system may also include a database and add the image of a biometric or 

portion thereof to the database. The system may store the image and the model of 
the image in the database. The image may be stored at full sampled resolution in the 
database or be compressed prior to being stored in the database. Preferably, if the 
image is compressed, the system compresses it in a lossless manner. The model 

20 may also be compressed prior to being stored in the database. The system may also 
encrypt the data or the model prior to storing them in the database. 

The system may also store additional information with the image and/or 
model in the database. For example, the associated information may include at least 
one of the following: identity of a person associated with the biometric; 

25 manufacturer, model, or serial number of the instrument supplying the data 

representing the biometric; the date and/or time of imaging the biometric; calibration 
data associated with the instrument used to acquire the biometric; temperature at the 
time the image was acquired; unique computer ID of the computer receiving image 
data from the instrument acquiring the image'of the biometric; or name of person 

30 logged onto the computer at the time the image was acquired. The associated 
information may also include a photograph, voice recording, or signature of the 
person whose biometric is imaged. The associated information may also be a 
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watermark, where the watermark may be identifying information (e.g., associated 
information as described above) or anti-tampering information to determine whether 
the image and/or model has been compromised. If compromised, the image and 
model are typically marked or removed from the database. 
5 The system may also include techniques for authenticating and/or identifying 

the person whose biometric is acquired. For example, the system may compare a 
previously stored model from a database to a present image, where the biometric is 
of a person having a known identity or an unknown identity. A "previously stored 
model" may be a model that has been stored, for example in a local or remote 

1 0 database, or is a model of a previously acquired image that has not been stored per 
se Similar usage of the "previously stored image" also applies herein. The present 
image may be a live image, an image previously stored in a local or remote database, 
a scanned image, or an image from another source, e.g., the National Institute of 
Standards and Technology (NIST) or Federal Bureau of Investigation (FBI) 

1 5 database. The system may compare the biometric features in at least two steps: 
comparing outline features and, if a candidate match is determined, comparing 
details features, and, if still a candidate match, then comparing pore features. In 
comparing outline features, the system may compare outline features of the 
previously stored model to outline features of the present image to determine (i) 

20 whether the present image is a candidate for a match or (ii) whether the previously 
stored model is a candidate for a match. In comparing the outlines features, the 
system may determine whether the comparison exceeds a predetermined candidate 
threshold. If the present image is not a candidate for a match, the system may 
compare outline features of a next previously stored model to the outline features of 

25 the present image to determine whether the present image is a candidate for a match 
and use the next previously stored model for details comparison if it is a match. If 
the previously stored model is not a candidate for a match, the system may compare 
outline features of a next previously stored model to the outline features of the 
present image to determine whether the next previously stored model is a candidate 

30 for a match and, if it is a match, the system may use the next previously stored 
model for details comparison. 
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If the system determines a candidate match of outlines features is found, the 
system may compare details features of the previously stored model with detailed 
features of the present image. The system may compare the details features by 
determining whether the details comparison exceeds a predetermined threshold or 
5 may determine whether required features associated with the previously stored 
model are found in the present image. In the case of biometrics related to friction 
ridge containing skin, the system may also determine whether pore features in the 
previously stored model are found in the present image. If so, the system may 
indicate which pores in the previously enrolled (i.e., acquired and modeled) image 

10 appear in the expected location in the present image, including allowance for 

distortions that normally occur between successive impressions, and may also show 
a pore count or a statistical probability of an error in such a match. The system, in 
comparing the outline, details, required details, and pore features, may determine 
whether the comparison meets a predetermined level of a number of consecutive 

1 5 frames in which the various features thresholds have been met, in order to call the 
comparison a match. The individual details and/or pores from successive frames 
need not be the same details and pores (unless specified as required) but could be 
different, but also exceeding the threshold(s). Further, the system may select another 
previously stored model for correlating with the feature set of the present image, and 

20 a successful match declared if any model or models exceed the threshold(s). 

The system may also scale and/or rotate the previously stored model(s) 
present image, or model of the present image for use in comparing the two. 

The system may also adaptively adjust the previously stored model(s) to 
account for variability associated with recording or acquiring the present image due 

25 to elasticity of the skin. For example, the variability may include stretching of the 
fingerprint, or portions thereof, laterally, longitudinally, or radially. The variability 
may also be caused by pressure of the fingerprint on a medium used to record or 
acquire the present image. In addition, this adaptive conformity may also take into 
account an expected location of ridge deviation details and, optionally, pore details. 

30 The system may also compare the previously stored model against multiple 

present images until a match is found or comparison with the multiple present 
images is complete. In another embodiment, the system may compare multiple 
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previously stored models against the present image until a match is found or 
comparison with the multiple previously stored models is complete. In yet another 
embodiment, the system may compare multiple previously stored models against 
multiple present images until a match is found or comparison among the multiple 
5 present images and the multiple previously stored models is complete. 

In some embodiments, the present image includes multiple fingerprints of an 
individual. For example, between two and ten fingerprint images of an individual 
may be captured and modeled. 

The system may also provide for preprocessing of the data representing the 

10 image. The preprocessing may include subsampling the image to capture the data. 
The preprocessing may also include decimating the data representing the image, 
where decimating may include removing every other row and every other column of 
the data. The system may also preprocess the data by binning the data, which 
includes averaging multiple "nearby" pixels to reduce the data to a predetermined 

15 size. The system may also correct for uneven imaging of the fingerprint, sometimes 
referred to as "flattening the field." Flattening the field compensates for light source 
properties, optics variations, Holographic Optical Element (HOE) variations, and 
differences among the gain and/or offsets of the pixels in a sensor array used to 
image the fingerprint. The system may also account for defective pixels in the 

20 sensor array, for example, by averaging pixel values around a defective pixel to 
determine a corrected intensity value of the defective pixel. The location and 
description of defective pixels may be provided by the manufacturer of the sensor 
array or measured during sensor assembly and stored in system memory for use 
during calibration of the fingerprint sensor. 

25 The preprocessing may also include encrypting the data representing the 

image. The preprocessing may also include changing the image orientation, such as 
horizontal or vertical flip and rotation, or a combination of the two. The system may 
also apply a watermark to the data representing the image or the system may attach 
acquire information, such as information about the instrument acquiring the image 

30 representing the biometric, to the data representing the image. The watermark may 
contain information and also may be used as a tamper-proofing technique, ensuring 
that a modified image is identified or not allowed to be used. 
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Another embodiment of the system according to the principles of the present 
invention includes an acquisition unit that acquires data representing an image of a 
biometric. The acquisition may be a biometric sensor to acquire live scan images, a 
photograph scanner to scan paper prints of biometrics, a computer modem to receive 
5 data from a database of biometric images, or other electronic medium performing 
similar functions. The system also includes a modeler that models features of the 
fingerprint utilizing at least two levels of image resolution. 

Various example embodiments of the instrument used to acquire images of 
biometrics are described herein. The embodiments may also include alternative 
10 embodiments, such as those disclosed in a related application, entitled "Acquisition 
of High Resolution Biometric Images," Attorney Docket No. 3174.1012-004, being 
filed concurrently herewith. The entire teachings of the related application are 
incorporated herein by reference. 

BRIEF DESCRIPTION OF THE DRAWINGS 
15 The foregoing and other objects, features and advantages of the invention 

will be apparent from the following more particular description of preferred 
embodiments of the invention, as illustrated in the accompanying drawings in which 
like reference characters refer to the same parts throughout the different views. The 
drawings are not necessarily to scale, emphasis instead being placed upon 
20 illustrating the principles of the invention. 

Fig. 1 is a computer network in which a fingerprint sensor according to the 
principles of the present invention is deployed; 

Fig. 2 is a system hierarchical diagram of the fingerprint sensor of Fig. 1; 

Fig. 3 A is a high level schematic diagram of the fingerprint sensor of Fig. 1; 
25 Fig. 3B is a detailed schematic diagram of the fingerprint sensor of Fig. 1; 

Fig. 4 is a generalized mechanical diagram of an imager in the fingerprint 
sensor of Fig. 1; 

Fig. 5 is an electrical schematic diagram of camera electronics in a camera of 
the fingerprint sensor of Fig. 1 ; 
30 Fig. 6A is a hierarchical diagram of a local computer used in the computer 

network of Fig. 1; 
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Fig. 6B is a block diagram of example software executed in the local 
computer of Fig. 1; 

Fig. 7 is a flow diagram of a process associated with acquiring and 
processing of the fingerprint image data acquired by the fingerprint sensor of Fig. 1; 
5 Fig. 8 is a high level block diagram of example processing elements 

operating in the computer network of Fig. 1 for processing the acquired fingerprint 
images; 

Fig. 9 is an image of a portion of a fingerprint acquired by the fingerprint 
sensor of Fig. 1; 

1 0 Fig. 1 0 is the image of Fig. 9 with an overlay of outline features of 

fingerprint ridges; 

Fig. 1 1 is the image of Fig. 9 with a subset of the fingerprint overlaid with 
detail features of the fingerprint ridges and deviations; 

Fig. 12 is a zoomed-in image of the portion of overlaid features of Fig. 1 1 ; 
15 Fig. 1 3 is a flow diagram of a fingerprint enrollment process executed by the 

enrollment software of Fig. 6B; 

Fig. 14 is a flow diagram of acquisition software of Figs. 6B and 13; 

Fig. 15 is a flow diagram of a process for constructing a model of the 
fingerprint acquired by the processes of Figs. 13 and 14; 
20 Fig. 16 is a flow diagram of fingerprint verification that is part of the analysis 

software of Fig. 6B; 

Fig. 17A-17C are flow diagrams of a matching processes beyond the outlines 
level executed by the fingerprint verification process of Fig. 16; 

Fig. 18 is a flow diagram of a fingerprint feature correlation process executed 
25 by the processes of Figs. 16 and 17; 

Fig. 19 is flow diagram of an adaptive conformity process executed by the 
fingerprint feature correlation process of Fig. 1 8; and 

Figs. 20A and 20B are diagrams including a set of pixels in the sensor array 
of Fig. 3 A that illustrates preprocessing for correcting bad pixels. 
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DETAILED DESCRIPTION OF THE INVENTION 

A description of preferred embodiments of the invention for a fingerprint 
biometric follows. It should be understood that the principles of the present 
invention and example preferred embodiments of the methods and apparatus 
5 described below may be applied to other biometrics, including: ear shape and 
structure, facial or hand thermograms, iris or retina structure, handwriting, and 
friction ridge patterns of skin such as fingerprints, palm prints, foot prints, and toe 
prints 

In general, the principles of the present invention include a procedure that is 

10 capable of identifying highly detailed fingerprint features by using gradient edge 
detection techniques at several image resolutions. The procedure identifies and 
traces edges of structural features to outline ridges and pores. At sufficiently high 
resolution, referred to in the industry as "level three details," the ridge outlines 
contain thousands of structural features that can be used in fingerprint matching. 

1 5 This capability improves matching reliability over systems that reduce ridge patterns 
to a few minutiae types or systems that consider ridges only as simple contour lines, 
via a process known as binarization or thinning. Because of the richness of features 
in fingerprints at high resolution, the procedure also allows for reliable matching of 
small portions or fragments of prints. In a preferred embodiment, edge detection 

20 software is combined with high resolution image acquisition technology that is 
capable of resolving pores and ridge profiles. 

Fig. 1 is a system diagram in which an embodiment of a fingerprint sensor 
100 according to the principles of the present invention is employed. The 
fingerprint sensor 100 includes a fingerprint imager 110 and fingerprint camera 120. 

25 The imager 1 10 and camera 120 may be mechanically, electrically, and optically 

connected in a single "box." A finger 105 is placed on the fingerprint imager 1 10 at 
a "viewable" location by the imager 1 10 for acquisition of a fingerprint 1 15 by the 
camera 120 and for modeling of the fingerprint 1 15 by processing as described 
hereinbelow. 

30 For many reasons, it is useful to design the fingerprint sensor 100 in as small 

a package as possible, such as for use in field operations, security systems, and other 
applications. However, although packaged in a small size, the fingerprint imager 
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110 and camera 120 are preferably designed in such a manner as to capture an image 
of the fingerprint 1 15 in high resolution. One way to achieve a small packaging size 
is through novel optical design. For example, the imager 1 10 may include a 
Holographic Optical Element (HOE). The HOE allows the fingerprint camera 1 20 
5 to be positioned close enough to the fingerprint 115 being imaged to receive, 
without use of large collecting optics, high resolution image features of the 
fingerprint 1 1 5 

Although a holographic optical element allows for minimizing the size of the 
fingerprint imager 110 and, consequently, the fingerprint sensor 100, the HOE is 

10 generally temperature sensitive. Therefore, compensating for the temperature 

sensitivity of the HOE is useful for acquiring accurate, high-resolution images of the 
fingerprint 115. Compensating for the temperature sensitivity of the HOE can be 
passive or active and is discussed further beginning in reference to Fig. 2. 

Continuing to refer to Fig. 1, the fingerprint camera 120 includes an interface 

15 to communicate bidirectionally with a local computer 130 via a control channel/data 
link 125. The fingerprint camera 120 sends image data 160 to the local computer 
130, and the local computer 130 may send control data 165 or other information, 
including image data 125, to the fingerprint camera 120 or imager 1 10 via the link 
125. 

20 The local computer 130 may include a variety of processing capabilities, 

such as modeling, authentication, and authorization, that is applied to the image data 
160. The local computer 130 may be in communication with a local database 135 
via a local link 132. Image data and associated model(s) 170, collectively, are 
communicated between the local computer 130 and local database 135 via the local 

25 link 132. Other data, such as administrative data, may also be communicated over 
the local link 132 for storage in the local database 135 for later retrieval. 

The local computer 130 may also communicate with a remote computer 150 
via a computer network 140, such as the Internet. The image data and associated 
model(s) 170 are communicated via network communications links 145 among the 

30 local computer 130, computer network 140, and remote computer 150. The remote 
computer 150 is in communication with the remote database via a remote database 
link 152. 
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The remote computer 150 may include some or all of the processing of the 
local computer 1 30 or include other services, such as remote retrieval of image data 
and associated model(s) 170 from a remote database 155 or authentication of a live 
scan image of a fingerprint. 
5 Fig. 2 is a hierarchical diagram of the fingerprint sensor 100. The fingerprint 

sensor 100, as discussed in reference to Fig. 1, includes a fingerprint imager 1 10 and 
fingerprint camera 120. Each will be discussed in turn. 

The fingerprint imager 110 includes, optics 210, and, optionally, active 
control circuits/element(s) 225. . The optics 210 includes a light source 205, optical 

10 elements 250, which are non-HOE's such as a waveguide and lens(es), and at least 
one HOE 410, which includes a hologram. 

The light source provides a collimated and expanded beam of light 207. The 
light source includes one or more beam shaping optical elements, and may include a 
coherent source, such as a laser diode, which works efficiently with a HOE, or a 

15 non-coherent source. 

The optional active control circuit/element(s) 225 may include an angle 
controller 230 and actuator 235. The actuator may be a Direct Current (DC) motor, 
stepper motor, piezo-electric actuator, or other electro-mechanical device capable 
and adaptable for use in moving the light source 205 at angles fine enough for use in 

20 the fingerprint sensor 100. A wavelength controller 240 may also be employed in 
the imager 1 10, where the wavelength controller 240 may be used to change the 
wavelength of the light source 205 in order to compensate for temperature-induced 
changes in the Bragg condition of the HOE. A power controller 245 may also be 
employed by the imager 1 10 to control the output power of the light source 205 for 

25 controlling exposure levels of the fingerprint 115. 

The fingerprint camera 120 includes a sensor array 215 and electronics 220. 
The sensor array 215 may be a Charge Coupled Device (CCD) or a Complementary 
Metal Oxide Semiconductor (CMOS), for example, and have a number of pixels 
providing a resolution fine enough for use in the fingerprint sensor 100. The 

30 electronics 220 are coupled to the sensor array 2 1 5 for receiving pixel data for 
processing. The electronics may include a processor, memory, and sensor data 
communications interface. 
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It should be understood that the hierarchical diagram of Fig. 2 is merely 
exemplary and could be configured in other ways and include additional or fewer 
components for implementing the principles of the present invention. 

Fig. 3 A is a generalized schematic diagram of the fingerprint sensor 100 and 
5 includes a subset of the components introduced in Fig. 2. The imager 1 1 0 of the 
fingerprint sensor 100 includes the light source 205 projecting a light beam 207 into 
the optics 210. An actuator 390 may be mechanically connected to the light source 
205, to the optics 210 or to both, directly or indirectly, to direct the light beam 207 
into the optics 2 10 in a controlled angular manner. Active control circuit(s) 225 
10 provide(s) control signal(s) 389 to the actuator 390 and/or the light source 205 in 
accordance with the descriptions above in reference to Fig. 2. The active control 
circuit(s) may receive feedback from the actuator 390 or light source 205 for control 
or regulation purposes. 

In this embodiment, a feedback signal 391 is presented to the active control 
15 circuit(s) 225 by the camera electronics 220. As in the case of typical feedback 
control systems, the feedback signal 391 is generated by the camera electronics 220 
as a function of a difference between an actual signal level and a desired signal level 
corresponding to imaging performance. In the case of the fingerprint sensor 100, the 
feedback signal 391 may represent a deficiency in light intensity emitted by the light 
20 source 205, or may represent an angular error of the light beam 207 projecting onto 
the optics 210, where the angular error may be caused by temperature effects on the 
HOE. The camera electronics 220 may determine the feedback signal 391 based on 
the image data 1 60, subset thereof, or other signal provided by the sensor array 215. 
Other photosensitive areas 380 outside the active pixel field of the sensor array 215 
25 may provide a signal 382, to the camera electronics 220, from which the feedback 
signal 391 is derived. 

The camera electronics 220 may also provide a control signal 165 to the 
sensor array 215 for use during imaging of the fingerprint features image 302. 
Further, the camera electronics 220 also includes an interface (not shown) for 
30 communicating with the local computer 130 via the communications link 125 for 
transferring the image data 160. 
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Fig. 3B is a detailed schematic diagram of the fingerprint sensor 100. A 
brief description of the imager 1 10 and camera 120 is described in turn. 

The imager 110 includes a power control circuit 245, angle control circuit 
230, and wavelength control circuit 240. The power control circuit 245 provides 
5 feedback signals to the light source 205 via an interface 393. Similarly, the 
wavelength control circuit 240 provides feedback to the light source 205 via an 
interface circuit 398. The angle control circuit 230 provides a signal to the actuator 
235 via an interface 396. 

The optics 210 includes optical elements 250 and at least one HOE 410. The 

1 0 optical elements 250 and HOE 4 1 0 are arranged in a manner adapted for imaging the 
features of the fingerprint 115. Details of the arrangement between the optical 
elements 250 and HOE 410 are described in detail beginning in reference to Fig. 4. 

Referring now to the details of the fingerprint camera 120, the electronics 
220 include multiple electrical components, including: logic 330, microprocessor 

15 335, microprocessor memory 355, system memory 345, interface circuit 360, and 
Analog-to-Digital Converter (ADC) 322, in embodiments where the sensor array 
2 1 5 outputs data in the form of analog signals. The microprocessor 335 may be 
integrated into the logic 330 or may be a separate component communicating with 
the logic 330 over a bus (not shown). The logic 330 may be a Field Programmable 

20 Gate Array (FPGA) or other logic device or a processor adapted for performing the 
functions described herein with regard to the logic 330. 

Communication between the sensor array 215 and the logic 330 occurs via a 
control interface 325, data bus 320, and, in certain cases, an alternate data line 385. 
Data is 'read out' of the sensor array 21 5 via the data bus 320 at a rate between 1 

25 MHz and 60 MHz, in some applications, but may be increased or decreased based 
on the application and technological limits. In this embodiment, an additional 
photosensitive area 380 outside the active pixel field of the sensor array 215 may 
provide a feedback signal 382 via the line 385 to the logic 330 for use in providing 
the power feedback 392 to the power control circuit 245, the angle feedback 395 to 

30 the angle control circuit 230, or the wavelength feedback 397 to the wavelength 
control circuit 240, or any combination thereof. The logic 330 may be designed to 
receive signals from a subset of pixels in the sensor array 215 for use in computing 
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an angle feedback signal 395, wavelength feedback signal 397, or power feedback 
signal 393, or any combination thereof. The logic 330 or microprocessor 335 may 
determine the feedback signals 391 (i.e., power feedback 392, angle feedback 395, 
or wavelength feedback 397) through use of various techniques, such as a Least- 
5 Means-Square (LMS) technique, optimization techniques, intensity differencing 
technique, or other process useful for determining single- or multi-variable control. 

Continuing to refer to Fig. 3B, the microprocessor 335 communicates to the 
microprocessor memory 355 via a microprocessor memory bus 350. The logic 330 
communicates with system memory 345 via a system memory bus 340. The system 

1 0 memory 345 may communicate with the interface circuit 360 via a memory/interface 
bus 365. The interface circuit 360 communicates with the logic 330 via a 
logic/interface control bus 370 and logic/interface data bus 375. The interface 
circuit 360 communicates with the local computer 130 via a local bus 125, which 
includes control lines 362 and data lines 364. 

1 5 In operation, the light source 205 produces an expanded, collimated optical 

beam 207 that is projected by the optical element 450 and HOE 410 to reflect off a 
cover plate 420 for imaging the features of the fingerprint 1 15 by the sensor array 
215. The sensor array 215 outputs data representing an image of at least a portion of 
the fingerprint 1 15 to the logic 330 via the data bus 320 at a sampling rate of, for 

20 example, 40 MHz. The logic 330 directs the image data 160 to different places in 
various embodiments. For example, in one embodiment, the logic 330 directs the 
image data 160 to the system memory 345 for additional processing or directs the 
image data 160 to the interface circuit 360 via the logic/interface data bus 375 for 
direct transmission to the local computer 130. The logic 330 may also direct a 

25 portion of the image data 160 to the microprocessor 335 for determining the 
feedback signals 391 in an embodiment in which active feedback control is 
employed in the fingerprint sensor 100. 

Fig. 4 is a schematic diagram of the imager 110. Referring first to the imager 
1 10, the light source 205 produces coherent, expanded beam of light 207 at a 

30 wavelength of, for example, 655 nm. The light beam 207 enters the optical element 
405 at an angle that causes the light 207 to be guided through the waveguide 405 by 
total internal reflection at the substrate-air interfaces. The light beam 207 
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encounters an interface between a Holographic Optical Element (HOE) 410 and the 
substrate waveguide 405, at which point, a portion of the light beam 207 is 
diffracted by the HOE, which includes a holographic grating, at a near normal angle 
to the guide surface and travels through a cover plate 415 to the finger 105. Fresnel 
5 reflection of the diffracted light at the interface of the cover plate 415 and fingerprint 
115, referred to herein as a "finger contact surface" ,directs some of the diffracted 
light back through the HOE 410, through the substrate waveguide 405, and onto the 
entrance face (i.e., sensor array 215) of the camera 120. Reflection at the cover plate 
is suppressed at locations where objects come into optical contact with the cover 

10 plate. The remaining reflected light carries an image of these contact areas to the 
camera 120. The image data 160, which represents an image of the fingerprint 1 15, 
is directed across the data bus 125 to the local computer 130 in a manner as 
discussed above in reference to Fig.l. 

One example embodiment of the fingerprint imager 1 10 is constructed as 

15 follows. The light source 205 is a laser diode that emits 5 mW of light at 652nm. 
The substrate (e.g., glass) waveguide 405 has an entrance face for the laser light 207 
that is beveled to an angle of about 60 degrees from the horizontal. The dimensions 
of the waveguide 405 are 36 mm long, 2.5 mm thick, and 25 mm wide. The cover 
plate is 1 mm thick and having a square surface of 25 x 25 mm. In this example, the 

20 image of the fingerprint 1 1 5 is captured by a CMOS electronic imager having a 
1024 by 1280 array of 6 urn square pixels and 256 gray levels. The size of the 
resulting image is 6. 1 mm by 7.7 mm, while its resolution is 167 pixel per mm or 
4200 pixels per inch. 

In operation, when the finger 105 is pressed onto the finger contact surface 

25 420, the ridges of the fingerprint 1 15 make optical contact and suppress reflection. 
Since pores are depressions along the ridges, there is reflection from the cover plate 
at pore locations. The resulting image of the fingerprint 115 that is presented to the 
camera 120 includes light colored areas for the valleys and dark ridges with light 
pores aligned along the ridges. An undistorted, high-resolution image of the 

30 fingerprint 1 15 can be captured by the camera if the light beam 207 that is diffracted 
by the HOE 410 is collimated and has a uniform wavefront. 
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Fig. 5 is a schematic diagram of a subset of the camera electronics 220. This 
schematic diagram provides a layout of the sensor array 215 and the FPGA 330. 
The size of these components 215, 330 results in a particular layout in which the 
finger 105 is positioned on top of the imager 1 10 in a manner such that the 
5 fingerprint image 500 acquired by the sensor array 215 may be mis-oriented in a 
way that inverts the left-right orientation of the image, for example, or rotates the 
image to a position clockwise from the desired viewing position, for example. The 
image is preferably, and in some cases must be, acquired and displayed in the same 
orientation as observed when inked and rolled on a white card, to preserve the 

10 traditional orientation used to store and compare pre-existing law enforcement 
image databases. Therefore, the FPGA 330 may be programmed to change the 
image orientation through predefined data manipulation. For example, the system 
memory 345 may be designed to store two or more times the amount of image data 
160 so as to allow the FPGA 330 to rewrite the image data within the memory in a 

1 5 manner for reading-out the data to the local computer 1 30, which allows the 

fingerprint image 500 to be displayed to a viewer in a standard "tip up" orientation. 
In the case of a large system memory, the size of the system memory 345 also 
allows for storage of multiple data size fingerprint images 500 for buffering or 
averaging purposes. It should be understood that the system memory 345 may be 

20 larger or smaller than just described depending on the particular application. 

Fig. 6A is a hierarchical diagram of the local computer 130. The local 
computer 130 includes a sensor data/control interface 605, software 610, display 
630, local storage 635, and network interface 640. The software 610 includes 
processing software 615, a Graphical User Interface (GUI) 620, the local database 

25 135, and sensor control software 625. The local computer 1 30 can be a standard 
computer that is customized to operate with the fingerprint sensor 100 or can be a 
custom-designed computer that includes specific, optimized hardware, such as a 
specific sensor/data control interface 605 to optimize performance with the 
fingerprint sensor 100. 

30 Fig. 6B is a block diagram of example processing software 615 that may be 

loaded and executed by the local computer 130. The processing software 615 may 
include acquisition software 645 (Fig. 14), enrollment software 650 (Figs. 13-15), 
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analysis software 655 (Figs. 16-20), administrative software 660 (e.g., database 
administration), and additional information processing software 665 (e.g., security 
features, header information, and watermarking). Details of the processing software 
61 5 are described in detail below in reference to the indicated associated figures. 
5 Fig. 7 is a system level flow diagram of a process 700 for (i) imaging the 

fingerprint 1 1 5 with the fingerprint sensor 1 00 through (ii) processing image data 
160 representing an image of at least a portion of the fingerprint. The process 700 
begins with the fingerprint sensor 1 00 capturing fingerprint image data (step 705). 
The image data 160 is presented to the fingerprint camera processor 335 or local 

10 computer 130 for preprocessing (step 710). Preprocessing can help speed system 
performance significantly, while maintaining more cost effective imager to 
computer interfaces that might otherwise be bandwidth limited. Preprocessing 710 
may include changing the image orientation, calibrating the image, scaling the 
image, flattening the field across the pixels of the sensor array 215, correcting for 

1 5 defective pixels of the sensor array 2 1 5, decimating the captured image data 1 60, 
applying encryption to the image data, applying a watermark to the image data, 
adding header information to the image data, or compressing the data, optionally 
through a lossless compression technique. Other forms of preprocessing may also 
be applied for transmission and/or storage with the image data 160. 

20 The image data and, optionally, the additional data (collectively 725) is provided to 
the fingerprint camera processor 335, local computer 130, or remote computer 150 
for processing the image data 160 (step 715). Examples of processing the image 
data 160 include: modeling the features of the fingerprint, enrolling the fingerprint, 
authenticating the individual associated with the data representing at least a portion 

25 of the fingerprint, and identifying the individual. The process 700 ends in step 720. 
Fig. 8 is a flow diagram of a process 800 for modeling the image as a 
function of the fingerprint features. The image data 160 is processed by a gradient 
edge detector 805. The gradient edge detector provides the image data and 
fingerprint features (collectively 815) to a modeler 810. The modeler generates a 

30 model of the fingerprint as a function of the fingerprint features. The modeler 810 
outputs image data and model(s) (collectively 170) to a local database 135 or other 
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processor, such as the remote computer 150 for additional processing or storage in 
the remote database 155, for example. 

Figs. 9-12 include images of fingerprints 115 that are (i) displayed to a user, 
(ii) modeled, (iii) processed (e.g., authenticated, identified, etc.), or (iv) stored for 
5 later retrieval. Figs. 13-20 include example processes that improve the analysis of 
both high-resolution fingerprints that contain resolved pores and lower resolution 
fingerprints without resolved pores. The processes of Figs. 13-20 do so by using 
identifying information in a fingerprint, which can include ridge shapes or profiles in 
addition to the traditional ridge contours, and the position, shape and prominence of 
10 pores. Before describing the processing, a brief description of the fingerprint images 
and overlays indicating fingerprint features determined by the processing is 
described. 

Fig. 9 is an example of a fingerprint image 900, represented by the image 
data 160, that was acquired with the fingerprint sensor 100. Light colored (e.g., 

1 5 gray) valleys (V) 905 and dark ridges (R) 9 1 0 are observable. Examples of pores 
(P) 915, ridges 910, ridge details (RD) 920, and minutiae (M) 925 are all indicated 
in the figure. Some or all of these types of features can be used by the processes of 
Figs. 13-20 to compare and match fingerprint images 900. The fingerprint image 
900 contains fewer than ten minutiae 925 and almost ten times as many pores 9 1 5 as 

20 minutiae 925. 

Fig. 10 is an example of the image data 160 including indications of features 
obtained at the "outline" resolution level. Thick lines 1005 trace identified 
boundaries between ridges 905 and valleys 910 and outline some of the pores 915. 
Features that are outlined in thick lines 1005 are considered real and are used for 

25 subsequent fingerprint matching steps. Noise is traced in thin lines 1010 and is not 
used for matching. A trained model of a single fingerprint includes one or more of 
these feature sets from various regions of the complete fingerprint image data 160. 

Fig. 1 1 is an example of the fingerprint image data 160 with a smaller central 
region 1 105 containing highlighted features that were identified using gradient edge 

30 detection procedures, discussed below beginning at Fig. 13. The features were 
obtained at the "details" level of resolution. Similar to Fig. 10, thick lines 1110 
indicate "real" (i.e., non-noise) features and thin lines 1115 indicate noise. 
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The local computer 130 may display the image data 160 in the form of the 
fingerprint image 900 in the GUI 620. Through use of standard or custom GUI 
techniques, the user can, for example, (i) select specific features that must be present 
for a match, (ii) move the modeled region 1 105 in any direction to model different 
5 area, (iii) enlarge or reduce the size of the modeled region, or (iv) select or deselect 
features within the region to reduce noise from being part of the model, or to include 
specific features within the region. 

Fig. 12 is an expanded view of the model region 1 105 from the fingerprint 
shown in Fig. 1 1 that includes indications of features or noise operated on by the 

1 0 user via the GUI 620. In the fingerprint image 900, the thick gray lines 1 1 1 0 are 

real, and the black lines 1 1 1 5 are noise. However, the user may want to change the 
results produced by the automated processing. For example, the user may use a 
Graphical User Interface (GUI) to deselect certain features 1205 he or she may 
consider noise even though the automated processing considered the features 1205 

1 5 real fingerprint features. The manual selection process may also work in reverse, 
where features the automated processing considers noise may be reselected by the 
user to be included as real fingerprint features 1210 through use of the GUI 620. 

There are thousands of features included in the enrolled model information 
set. A later comparison to these features that finds, for example, 80% of the total 

20 enrolled features means that 4000 separate features were found that match an 
original enrolled image of 5000 total features in an area as small as 5x5mm, for 
example. 

In addition, the user may define specific features 1215, or a percentage of 
specific features as being required, in addition to a percentage of all features, as 
25 qualification criteria for a match. The more features 1215 that are required for a 
match reduces the likelihood of false detections but increases the likelihood of 
missed detections. Therefore, a threshold number of required features 1215, such as 
80%, may be specified to account for noise, distortions, lighting or other imaging- 
related effects. 

30 Multiple biometrics (e.g., multiple fingerprints of an individual; 

fingerprint(s) and palm prints; fingerprint(s) and iris scan; and so forth) may be 
acquired and modeled. The multiple biometrics may be combined in various 
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manners, such as root-mean-square or weighted sum, to for a "combined metric." 
The combined metric may be used for later comparing of the acquired multiple 
biometrics with future acquired images (e.g., live scans). Combining multiple 
biometrics may improve system reliability. 
5 The image data and model 170, collectively, may be transmitted as a paired 

set from the local computer 130 to the local database 135 or remote computer 150 as 
discussed above in reference to Fig. 1 . In applications where the smallest possible 
model is required, on a smart-card or drivers license, for example, only the model, 
without the accompanying image may be used. The outline features, details features, 

1 0 required features 1215, and fine details (e.g., pores) features may be stored as part of 
the model and used during later processing. 

Referring now to processing aspects of the present invention, two major 
steps are described: fingerprint enrollment (Figs. 13-15) and fingerprint matching 
(Figs. 16-19). During enrollment, a high-resolution digital fingerprint (e.g., image 

15 data 160 representing the fingerprint image 900) is acquired, and one or more sets of 
features to be used for subsequent fingerprint matching are identified. Fingerprint 
matching compares these sets of features (e.g., ridge deviation details, such as the 
required features 1215, pores, and minutiae) to the unknown fingerprints and decides 
if the two fingerprints are from the same individual. The source for these two 

20 fingerprints can be a database of high-resolution fingerprints, a live scan image, a 
legacy database of lower resolution fingerprints, latent fingerprints (i.e., fingerprints 
"lifted" at a crime scene), or combinations of these sources. Although the invention 
makes use of the extensive data available with higher resolution images, the 
techniques also work with lower resolution images. 

25 The processes of Figs. 1 3-20 describe an automatic fingerprint recognition 

method that identifies fingerprint features by using gradient edge detection 
techniques. In one embodiment, the features are acquired in gray-scale. In another 
embodiment, color may be part of the acquired image. The techniques identify and 
trace edges of structural features to outline ridges and pores. At sufficiently high 

30 resolution, the ridge outlines contain thousands of unique structural features (i.e., 
level three features) that can be used in fingerprint matching. This capability 
improves matching reliability over systems that reduce ridge patterns (by 
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binarization and thinning) to a few minutiae types Because of the richness of 
features in fingerprints at high resolution, the method and apparatus according to the 
principles of the present invention produces reliable, high speed matching of small 
prints to a much higher degree of accuracy than with standard minutiae-only based 
5 systems. 

The processes of Figs. 13-20 analyze an unaltered gray-scale image of the 
fingerprint 1 15 to reduce both computing time and false accept rates. In a preferred 
embodiment, edge detection software is combined with high resolution image 
acquisition technology that is capable of resolving pores and ridge profiles. 

10 Fig. 13 is a flow diagram of an example process 1300 for enrolling a 

fingerprint 115. The fingerprint 1 15 is first acquired (step 1305). An example 
process for acquiring an image is provided in Fig. 14. 

Fig. 14 illustrates a flow diagram of an example process 1400 for acquiring 
an image of the fingerprint 1 15, described in reference to Fig. 4. A direct light beam 

1 5 207 is transmitted through a substrate waveguide 405 (step 1405). The finger 105 is 
pressed onto a contact area of a coverplate (step 1410). The exposure time of the 
fingerprint camera 120 is adjusted (step 1415) either manually or automatically. The 
fingerprint image 302 (Fig. 3A) is captured (step 1420), and the process 1400 returns 
(step 1425) to the process 1300 of Fig. 13 for further processing. 

20 Referring again to Fig. 13, after acquisition, the image data 160 is transferred 

(step 1310) to the local or remote networked computers 130, 150, respectively, by 
one of a number of possible connecting protocols, such as, but not limited to, IEEE 
1394 (Firewire), USB, Serial ATA, fiber-optic cable, any applicable wireless 
communications protocols, such as BlueTooth or 802.1 lg, or Ethernet. The image 

25 data 1 60 is then converted (step 1 3 1 5) to a desired file format, and header 

information, and optionally a watermark, may be added to the image data 160. The 
header may contain information regarding the fingerprint 1 1 5 and how, when, and 
where it was acquired, the manufacturer, model and serial number of the acquisition 
device, the unique ID and name of the computer as well as the name of the operator 

30 logged on to the operating system, and pointers to related information, such as 
personal information about the individual, including, but not limited to, a 
photograph, voice recording (sometimes referred to as a "voice print"), signature, or 
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other identifying information. A watermark can be added to protect the image and 
data header from subsequent unauthorized alteration. One example of a procedure 
to add a watermark to fingerprint images is described by Yeung and Pankanti, 
"Verification watermarks on fingerprint recognition and retrieval" Journal of 
5 Electronic Imaging, 9(4), 468-476(2000). Commercial watermarking procedures 
can also be used. 

Continuing to refer to Fig. 13, after the fingerprint is acquired (step 1305), 
specific and unique features of the fingerprint are identified, extracted and stored as 
a "trained model" (step 1320), according to a process described below in reference 

10 to Fig. 1 5. The model may be used for subsequent fingerprint verification (one-to- 
one matching) by comparing the feature sets of the model to the corresponding 
features of a subject, or "present," image. Features from the entire image are usually 
extracted and stored for fingerprints that are to be used for fingerprint identification 
(one-to-many matching). An example process of constructing a model is described 

1 5 in reference to Fig. 15. 

Fig. 15 is a flow diagram of an example process 1500 for creating a model of 
a fingerprint that can be used for subsequent analysis and matching. At least two 
models are constructed (step 1505) from high resolution images: a low resolution 
"outline" model that comprises ridge contours, and a high resolution "details" model 

20 that comprises ridge contours, ridge shapes, and pores. Details regarding ridge 

shapes and pores are generally not considered in the outline model; compare Fig. 10 
for an example of the features that are typically identified at the outline level to Figs. 
11 and 12 for features found at the details level. A further refinement adds a third, 
more highly detailed model specifically designed to more closely identify pore 

25 feature information, such as area, and centroid location. Edge detection levels may 
be changed for each of the three models in order to identify specific details, such as 
pores within ridges, with higher accuracy. 

Since the outline model contains less information, computation time is 
reduced by first comparing images at the outline level for a candidate match before 

30 attempting to match candidate images at the details level. Both outline and details 
models may be constructed using the same procedure of gradient edge detection 
(e.g., gray-scale gradient edge detection) (step 1515). In one embodiment, the 
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outline model is determined from a subset of pixels of the original image; each pixel 
of the subset is an average value over a predetermined number of neighboring pixels 
from the original image. The first step during fingerprint matching is to compare 
outline models. Matches may be discovered at the outline level and then compared 
5 at the details level. Detailed matches may then be supplemented by required details, 
and then further to be compared at the fine details pore level. 

Images acquired by the fingerprint imager 1 10 of Fig. 4 are 6.1 mm by 7.7 
mm, in one embodiment. This relatively large fingerprint is usually divided into one 
or more smaller regions that are individually used to construct individual model(s) 

10 (step 1510). These model regions can either be chosen manually to capture 

particularly interesting features found in a specific region, or they can be chosen 
automatically using adjustable software settings that define the number of regions, 
region size, and region locations. Their size is preferably chosen to be large enough 
to include numerous characteristic features and small enough to reduce problems 

15 associated with plastic deformation of the skin and to minimize computation time by 
using a smaller relative model size. The features identified in a sub-region of the 
complete fingerprint are referred to as a "feature set." A trained model comprises a 
collection of all of the feature sets for a particular fingerprint, or several trained 
models may each contain feature sets for a portion of the particular print 

20 Features, for each resolution level and in each region of the fingerprint 

chosen to be part of the trained model, are identified using gray-level gradient edge 
detection procedures (step 1515) and extracted. The gradient is first estimated for 
each of the pixels of the model using one of a relatively large number of procedures 
that have been developed for this process (see for example D. A. Forsyth, and J. 

25 Ponce, "Computer Vision A Modern Approach", Prentice Hall, New Jersey, 2003, 
chapter 8,). 

A particularly useful procedure that is often used to estimate gradients is to 
apply a Gaussian noise filter to the image and then to perform the gradient 
calculation using a "finite differences" algorithm. After calculation of the gradients, 
30 an image point with a locally maximal gradient in the direction of the gradient is 
identified and marked as an edge point. The next step is to identify neighboring 
edge points. This is usually accomplished by finding the nearest pixels to the 
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original edge point that lie in a direction that is approximately perpendicular to the 
gradient direction that passes through the first edge point. The gradient values for 
these new pixels are determined, and the pixel with a gradient that is (i) maximal 
along its gradient direction and (ii) has a value that exceeds a threshold is assigned 
5 to be the next edge point. This procedure is continued either until the edge is 
terminated or the edge closes with itself to form a continuous curve. Edge 
termination occurs at the previously determined edge point if the gradient of a 
candidate edge point is less than the threshold. In the next step a previously 
unvisited edge point is identified and its edge is traced according to the steps 

10 outlined above. This whole process is repeated until all of the potential edge points 
have been considered. Automatic software procedures are then used to distinguish 
fingerprint features from noise. Real edges, for example, must define features that 
have a minimum width. In addition, lines that do not enclose pores must extend for 
a minimum distance in order to be considered as legitimate feature edges. Further, 

15 pores only occur in ridges and not in valleys. Additional rules may be applied by 
adjusting software settings. Optional steps allow the models to be manually edited 
by adding or deleting features (step 1520) and allow users to indicate certain features 
of the model that must be present for a successful match (step 1525). All editing is 
performed on the features and not on the original image, which is deliberately 

20 protected from any alteration. Examples of features that are identified by this 
procedure were introduced above in reference to Figs. 10-12. A number of 
commercial software applications can be used for edge detection, or custom 
software applications may be designed for executing the processing described 
herein. The process 1500 returns (step 1530) to the enrollment process 1300 of Fig. 

25 13. 

Referring again to Fig. 13, after feature extraction to produce the trained 
model, the original fingerprint image and model data 160 is optionally indexed, 
compressed and encrypted (step 1325), and stored with the model (step 1330). One 
compression procedure is the Federal Bureau of Investigation (FBI) standard for 500 
30 dpi fingerprints, referred to as Wavelet/Scalar Quantization (WSQ). This is a lossy 
scheme with a compression ratio of 12.9. According to the FBI standard, the 
compressed image needs to form a record of the original print that can be used for 
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manual matching and for some automatic matching procedures. Alternatively, a 
lossless compression technique may be employed for storing the image. The trained 
model used for matching, on the other hand, requires much less storage space and 
may be stored with or without compression. 
5 Fingerprint matching is used either to verify the identity of an individual 

(referred to as 1 :1 matching) or to identify the source of an unknown fingerprint 
(referred to as 1 :n search). Both procedures compare a known fingerprint from a 
database either to a live scan fingerprint for verification or to an unknown 
fingerprint for identification. This fundamental step of fingerprint matching is the 

1 0 same for both verification and identification. 

Fingerprint verification is a particularly important type of fingerprint 
matching. Fingerprint verification compares a live scan image to an enrolled image 
in order to authenticate the identity of the individual presenting the live scan 
fingerprint. Flow diagrams for fingerprint verification are shown in Figs. 16,17, 18 

15 and 19. 

A user first presents identification (step 1605), such as a passport, license, 
smartcard or other ID, name or password any of which may be used to provide or to 
look up his/her previously enrolled fingerprint model(s). Note that the models may 
be looked-up or stored on the ID, depending upon the application. Then, a live scan, 

20 high-resolution fingerprint of the user is acquired in real time (step 1 620), and 

"outline," "details," and "pores" features for the complete fingerprint are extracted 
(step 1625), as illustrated in reference to Figs. 9-12. 

Fig. 16 is a flow diagram of the initial process 1600 for fingerprint 
verification at an "outline" resolution level. An outline feature set from the trained 

25 model (steps 1610, 1 6 1 5) of the user is compared to the corresponding features from 
the live scan image (step 1630) according to a process described below in reference 
to Fig. 18. This comparison is repeated until either a match is found or all of the 
feature sets of the model have been compared to the live scan outline features (steps 
1635, 1640, 1645). If no match is found, the process 1600 ends (step 1655), and 

30 further processing at the details level does not occur for that particular model. If a 
match candidate is found at the outline level (step 1635), the matching procedure is 
repeated at the details level (process 1650) according to Figure 17A. 
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Fig. 17A is a flow diagram of the process 1650 for fingerprint verification at 
the "details" resolution level. The process 1650 selects a details feature set from the 
trained model (step 1705). The process 1650 compares the details features from the 
image and model (step 1710) according to the process of Fig. 18, discussed below. 
5 This comparison is repeated until either a match is found or all of the feature sets of 
the model have been compared to the live scan details features (steps 1715, 1720, 
1725). If no match is found, the process 1650 ends (step 1735), and further 
processing at the required features level does not occur for that particular model. If 
a match candidate is found at the details level (step 1715), and no "required" or fine 

10 details "pores" features are specified (i.e., details threshold is met), then a positive 
match is declared. If a match candidate is found at the details level (step 1715) and 
"required" and/or "pores" are specified in the matching requirements, the matching 
procedure is repeated for required features (process 1730) according to Fig. 17B. 

Fig. 17B is a flow diagram of the process 1730 for fingerprint verification of 

15 [the optional] required features. The process 1730 selects required features from a 
feature set from a trained model (step 1 732). The process 1 730 compares the 
required features from the feature set of the model to features identified in the live 
scan image using the same resolution that was used to obtain the required features of 
the model (step 1734), according to the process of Fig. 18, discussed below. The 

20 comparison is repeated until either a match is found or all of the feature sets of the 
model at the required features level have been compared to the live scan image 
(steps 1736, 1738, 1740). If no match is found, the process 1730 ends (step 1745), 
and further processing for pore matching does not occur for that particular model. If 
the required features match (step 1736), the matching procedure is repeated for pores 

25 at the "fine details" level (if any, process 1750) according to Fig. 17C. 

Fig. 17C is a flow diagram of the process 1750 for fingerprint verification of 
the optional pores at the "fine details" resolution. The process 1750 selects pores 
from a feature set from a trained model at the fine details resolution (step 1 752). The 
process 1750 compares pores from the feature set of the model to pores identified in 

30 the live scan image (step 1 754) according to the process of Fig. 1 8, discussed below. 
The comparison is repeated until either a match is found or all of the feature sets of 
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the model at the fine details level have been tested. If the pore features threshold is 
met, a positive match is declared (step 1770). 

Fig. 18 is a flow diagram outlining the process used to determine if a match 
exists between a trained model and a present image, such as a live scan fingerprint 
5 image, which is used now as an example. Fingerprint features are correlated at 
either the outline, details, or fine details resolutions (steps 1630, 1710, 1734, 1754). 
A model feature set is overlain on a feature set of the entire live scan image (step 
1805). Since the model feature set is generally smaller, it will overlap only a portion 
of the live scan feature set. A correlation is then calculated between the model 

10 feature set and the portion of the live scan image that it overlays (step 1810). There 
are a number of procedures that are commonly used to determine correlation 
between images (for example, Maltoni, Maio, Jain, and Prabhakar, "Handbook of 
Fingerprint Recognition", Springer, 2003, chapter 4). A particularly useful 
procedure considers the similarity of two images to be indicated by their "cross 

1 5 correlation" (for example, Maltoni, Maio, Jain, and Prabhakar, "Handbook of 
Fingerprint Recognition", Springer, 2003, chapter 4). Using this procedure, the 
cross correlation between the live scan image and the model feature set is calculated 
as the feature set expands, contracts, and rotates, over predetermined limits, with 
respect to the live scan image that it overlays. The feature set is then translated to a 

20 new section of the live scan image (step 1 825), and the cross correlation calculation 
is repeated. This process is followed for all the feature sets of the model (step 
1820). A match is achieved (step 1830) if a cross correlation exceeds a threshold 
(step 1815), and if a predetermined percent of features match with the live scan 
image features (step 1830). Otherwise, there is no match (step 1840). 

25 Fig. 19 is an "adaptive conformity" process 1900 optionally executed in 

determining the correlation (step 1810). The adaptive conformity process 1900 is 
used to improve performance of the correlation process of Fig. 18. Adaptive 
conformity allows for matching of fingerprint features in cases where linear or non- 
linear shifting of the fingerprint features occurs either in the original fingerprint 

30 from which an outline, and/or details, and/or fine details level model(s) is derived, or 
in imaging a live scan fingerprint image. Such linear or non-linear shifting is caused 
by elasticity of the skin in combination with pressing the finger 105 into the imaging 
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surface (e.g., lens 415) in various ways, such as through applying pressure in 
latitudinal, longitudinal, or rotational directions or in an excessively hard or light 
manner. In the latitudinal, longitudinal, or rotational cases, part of the fingerprint 
may be imaged normally, and compression, extension, or rotation may occur in 
5 another part of the fingerprint. In the excessively hard or light pressure cases, part 
or all of the fingerprints may include thicker or thinner ridge 905 sizes, respectively. 
The adaptive conformity process 1900 allows the shape of the model to adaptively 
conform to account for non-uniformity of fingerprinting but in a manner that does 
not alter or change the features in any way, thereby maintaining the integrity to the 

10 identification and verification processes according to the principles of the present 
invention. Similar shapes, details, and pores that are shifted slightly due to distortion 
are recognized as the same shapes, but in a slightly different relative position than 
they were on a previous acquisition. The principles of the present invention allow 
for highly accurate recognition of level three details even if distortion from previous 

1 5 enrollment has taken place, which is normal to some degree in any set of fingerprint 
images. In addition, better accuracy to match is obtained with slight changes due to 
wear of ridges, scarring, dehydration, and pressure. 

Referring specifically to Fig. 19, the adaptive conformity process 1900 may 
be executed for authentication or identification (i.e., verification) processes, and 

20 with any image source including but not limited to latent images. After starting 
(step 1905), the process 1900 returns to the fingerprint feature correlation process 
1800 of Fig. 18 if adaptive conformity is not selected to be applied by the user (step 
1920). In one embodiment, the user may select adaptive conformity to be selected 
through toggling a Graphical User Interface (GUI) control. If adaptive conformity is 

25 selected, the process 1900 identifies a candidate match start location (step 1915), 
such as the core (i.e., center of fingerprint "swirls"), and uses the start location as a 
"base" from which to apply the adaptive conformity linear or non-linear processing. 

After identifying the start location, the process 1 900 locates the pixels where 
edge topology (i.e., outline level) features of the model and the image deviate. 

30 Using the deviation point as a "base point of adaptive conformity," the process 1900 
attempts to conform potentially corresponding model pixels beyond the base point 
laterally, longitudinally, or radially (step 1930) with the fingerprint features of the 
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live scan fingerprint image. Conforming the potentially corresponding model pixels 
means to shift the model pixels in a predetermined direction without changing the 
shape of the outline or details features of the model. If the chosen predetermined 
direction is correct, the shifting of the edge topology continues by shifting pixels in 
5 the same direction from the base deviation while the shape of the edge topology 
continues to match (steps 1935, 1940, and 1945) or until the edge topology being 
examined is complete. If the chosen predetermined direction is incorrect, other 
directions may be tried. Distance ranges for allowing pixels to move in attempting 
to adaptively conform can be specified by the user but are limited in range to that 

1 0 typical of these types of distortions. 

Additional testing at the outline level continues (step 1950) until the outline 
feature sets of the model are compared with the outline features of the live scan 
image (steps 1950 and 1925-1945). Following comparison of the outline features, 
the process 1900 repeats the adaptive conformity process at the details level (step 

15 1955). The process 1900 returns to the process of Fig. 18 at step 1810 following 
completion (step 1960) at the details level. In an alternative embodiment, if the 
matching at the outlines level does not achieve a predetermined threshold, the 
processing at the details level (step 1955) is skipped with an appropriate message 
being passed to the process of Fig. 8. 

20 Fingerprints from an unknown individual can sometimes be identified by 

comparing the unknown print to a database of known prints; this is referred to as 
1 :n fingerprint identification. A particularly advantageous procedure is to compare 
fingerprints in a database of high resolution fingerprints that were acquired and 
processed according to the processes of Figs. 13-15 with a high resolution unknown 

25 fingerprint. In this case, each fingerprint in the database has an associated "outlines" 
and "details" and optionally, a "required" and "pores" feature set that encompasses 
the entire fingerprint image. A trained model is formed from the unknown 
fingerprint and compared to the feature sets of the database. To save computation 
time, the comparison is first made at the "outline" resolution level. The subset of 

30 candidate fingerprints that match at the outline resolution level is subsequently 
compared at the "details," and optionally at "required" or "pore" resolution(s). 
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It is also possible to compare a relatively low resolution unknown print to 
prints in either a high or a low resolution database. In these cases, the feature set for 
the unknown fingerprint includes only ridge patterns and does not contain 
information on ridge profiles, ridge shapes, or pores. The processes described herein 
5 exhibit enhanced reliability over minutiae-based systems even in this case since all 
of the information of the fingerprint is used for comparison and not just a few 
minutiae points. 

In most examples of fingerprint identification, appropriate linear scaling 
might be necessary since the unknown fingerprint may have been acquired at a 

10 different magnification from the fingerprints in the comparison database. The GUI 
620 allows a user to graphically mark a line having start and end points of a scale 
(e.g., ruler) imaged with the original fingerprint and assign a length value to the 
length of the line. In this way, proper scaling can be applied for comparison against 
a live scan image, for example. 

1 5 Similarly, appropriate angular rotation might be necessary since the 

unknown fingerprint may have been acquired at a different angle than the 
fingerprints in the comparison database. The GUI allows a user to indicate a range of 
rotation (e.g., + or - 30 degrees) to check the live scan image against the outline 
level of the comparison model. In this way, proper orientation can be obtained for 

20 the comparison, and subsequent details, required, and pore features can then be 

applied at the same rotation, for example. Checking a larger degree of scale and/or 
rotation takes more computing time, so faster compare times are achieved if some 
degree of normalization is first preprocessed in order to limit the degree of scaling 
and/or rotation that is required for reliable operation of the system. 

25 In addition to the processing described above, the principles of the present 

invention support preprocessing that can improve performance of the processing 
(i.e., modeling and comparisons). Example forms of preprocessing include: image 
orientation and rotation, sub-sampling, decimating, binning, flattening the field, 
accounting for defective pixels in the sensor array 215, encrypting the image data 

30 160, applying a watermark, and attaching sensor information. 

Figs. 20A and 20B are illustrations of one form of preprocessing, namely 
accounting for defective pixels. Referring first to Fig. 20A, a grid 2000 illustrates an 
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example of a small region of pixels in the sensor array 215. The processing region 
of interest 2005 starts at a first position. In this embodiment, the processing region 
of interest is composed of an array of 3x3 pixels. Alternative embodiments may 
utilize alternative array sizes (5x5, 9x9, etc) or alternative array shapes (rectangular) 
5 ultimately determined by the desired resolution of the fingerprint sensor and the 
desired image quality. In the present example, there are multiple bad pixels in the 
sensor array 215, each represented by a zero '0'. Among the pixels, there are known 
good pixels represented by a one ' F. In the center of the processing region of 
interest 2005 is a pixel of interest, represented by a plus sign '+'. In this example, 

1 0 the pixel of interest, if a defective pixel, may be corrected by obtaining an average of 
at least two neighboring pixels within the processing region of interest 2005. The 
average value of the neighboring pixels can then be assigned to the defective pixel 
of interest. Once a pixel of interest has been corrected (if necessary), the processing 
region of interest 2005 is moved by one pixel spacing in order to address the next 

1 5 pixel in the array. The direction of movement of the processing region of interest 
2005 may be in the most convenient direction for processing efficiency or dictated 
by the architecture of the particular system. 

Fig. 20B is an example of the shifting of the processing region of interest 
2005 to a neighboring pixel on the pixel array 2000. Noting that the pixel of interest 

20 previously corrected in Fig. 20A is now denoted as a 'C pixel in Fig. 20B, the 

processing region of interest corrects the next pixel of interest (also donoted by a + 
in Fig. 20B) by averaging two known good pixels from within the processing region 
of interest 2005. In alternative embodiments, previously corrected pixels (denoted 
as 'C in Fig. 20B) may be used as a good pixel for the purposes of correcting the 

25 new, defective pixel of interest. 

Although this embodiment averages the intensities of two known good pixels 
within each processing region of interest to correct for a defective pixel, alternative 
embodiments may average the intensities of more than two known good pixels. In 
yet another embodiment, a defective pixel of interest may be replaced by intensity 

30 data from one known good pixel from within the processing region of interest 2005, 
wherein the processing region of interest 2005 may be of an array size larger than 
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3x3 or the processing region of interest array size may not be a square two- 
dimensional array. 

The other examples of preprocessing are now described without reference to 
associated figures. 

5 Subsampling is a form of reducing the amount of data sampled by the 

fingerprint camera 120. Instead of reading data from every pixel of the sensor array 
215, the camera 120 reads a subset of the pixels in a predefined manner, which is 
preferably selectable by the user. Decimating is another form of reducing the data, 
but reduces the amount of data after the data has been read from all or substantially 

10 all the pixels of the sensor array 215. Binning is a technique for reducing the 

amount of data representing the fingerprint by averaging data of multiple pixels into 
a single value; for example, four pixels may be averaged as a single average value to 
reduce the data by a factor of four. 

Flattening the field is also referred to as correcting for uneven illumination of 

1 5 the fingerprint image. Uneven illumination of the image may be caused by many 
sources, such as the light source 205, the propagation path of the light beam 207, the 
optics 210 (including the optical elements 250 or HOE 255), or the gain or offset of 
the pixels of the sensor array 215. Testing for uneven imaging of the fingerprint 
may be done during a calibration phase prior to acquiring an image of a fingerprint, 

20 where a reflection off the fingerprint imaging surface is used as the source and 
calibration data is determined and stored in the microprocessor memory 355 (Fig. 
3B), for example. The calibration data is applied during live scanning of a 
fingerprint 115. 

In addition to reducing data or correcting for imaging or equipment issues, 
25 the preprocessing may also be used to apply additional information to the image data 
160. For example, the image data 160 may be encrypted using various forms of 
well-known encrypting techniques. A header may be applied to add a variety of 
equipment-related information or other information that may be useful to determine 
the source of the equipment or understand the environmental conditions that were 
30 present when the fingerprint was imaged for use at a later time, such as during 
authentication or authorization. Example information may be the manufacturer, 
model and serial number of the instrument supplying the image data 160 
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representing the portion of the fingerprint, date of fingerprinting, time of 
fingerprinting, calibration data associated with the instrument used to acquire the 
image, the name of the operator logged onto the operating system, the unique ID of 
the computer that was used to acquire the image, and temperature at the time the 
5 image was acquired. 

A watermark may also be associated with the data. An example technique 
for applying a watermark is to use extra data bits associated with each pixel to 
represent a portion of the watermark. For example, if each data pixel only 
represents two hundred fifty-six (2 8 ) levels of gray-scale, two bits of a ten-bit word 

1 0 may be used to represent a portion of a watermark. The watermark may be 

information that would otherwise be part of a header or may be information that is 
used to determine whether any tampering has occurred with the fingerprint image. 

The processing and preprocessing discussed herein may be implemented in 
hardware, firmware, or software. In the case of software, the software may be stored 

1 5 locally with a processor adapted to execute the software or stored remotely from the 
processor and downloaded via a wired or wireless network. The software may be 
stored in RAM, ROM, optical disk, magnetic disk, or other form of computer 
readable media. A processor used to execute the software may be a general purpose 
processor or custom designed processor. The executing processor may use 

20 supporting circuitry to load and execute the software. 

While this invention has been particularly shown and described with 
references to preferred embodiments thereof, it will be understood by those skilled 
in the art that various changes in form and details may be made therein without 
departing from the scope of the invention encompassed by the appended claims. 

25 For example, in Fig. 1, the fingerprint sensor 100 and local computer 130 are 

distinct devices with a majority of the processing occurring in the local computer 
130. In an alternative embodiment, the fingerprint sensor 100 includes an embedded 
processor capable of performing preprocessing as well as some of the processing 
functions of the local computer 130, as discussed in general in reference to Figs. 6A 

30 and 6B and in detail in reference to Figs. 13-20. Also, the processing may be done 
in the remote computer 1 50. 
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The control channel/data link 125 or other links 132, 152, 145 may be wired 
or wireless links, such as through Radio Frequency (RF) or infrared 
communications. In another embodiment, the fingerprint sensor 100 may have IEEE 
802.1 1, cellular communications (e.g., Code Division Multiple Access (CDMA)), or 
5 other wireless capability to interface directly to a wireless node (e.g., base station, 
not shown) in the computer network 140. 

The local and remote databases 135, 155 may be any form of database and 
located with or distinct from the associated computers or distributed about the 
computer network 100. There may also be security provisions associated with the 

10 databases 135, 155 so as to prevent tampering with the fingerprint image data and 
models 170 stored therein. 

In Figs. 3A and 3B, real-time automatic feedback control of power, angle, or 
wavelength is illustrated. In alternative embodiments, mechanisms to allow for 
manual adjustments may be provided. Also, periodic, random, or calibration period 

1 5 adjustments instead of real-time feedback control may be employed to reduce power 
consumption. The power source for the fingerprint sensor 100 is described above as 
being the local computer 130 via Firewire or other interface; however, it should be 
understood that the power source may be a battery (not shown) or AC-to-DC power 
converter with sufficient filtering circuitry to achieve the high-resolution images. 

20 Also, the sensor array 2 1 5 may be a CCD array or any other array adapted for use in 
the fingerprint imaging application for achieving resolution sufficient to detect the 
fingerprint features described herein. 

In Figs. 6A and 6B, the software 610 may be any language adapted to be 
executed in the fingerprint sensor 100, local computer 130, or remote computer 150. 

25 For example, the software may be assembly language, 'C, object-oriented C++, 
Java, or combinations thereof. It should be understood that the processing and 
display of live scan images should be displayed in near-real-time, preferably with 
outline or details models displayed thereon. 

Fig. 7 may omit the preprocessing 710 and processing 715 of the image data 

30 160. Instead, the image data 160 may be stored in one of the databases 135, 155, for 
example, and processed according to the preprocessing 710 or processing 715 
techniques as described herein in a post-processing, non-real-time manner. 
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Additionally, in another embodiment, after images are captured by the fingerprint 
sensor 100 and indexed and stored, certain software-based enhancements may be 
performed, if needed, at the option of a system configuration administrator or for 
other beneficial reasons. Certain enhancements that do not alter the original image 
5 or its unique characteristics can be performed to enhance image analysis, such as 
mean low-pass filtering or automatic level adjustment to improve contrast or other 
methods that include, but are not limited to, gray-scale gradient edge detection 
techniques for pattern recognition and subsequent matching techniques, or 
combinations thereof. 

1 0 A number of commercial software applications can be used for edge 

detection, including Aphelion from Amerinex Applied Imaging, Hexsight software 
from Adept, Vision Blox distributed in the U.S.A. by Image Labs, and Halion from 
The Imaging Source. 

Fig. 8 illustrates a gradient edge detector 805 and modeler 810 as distinct 

15 operational units. In alternative embodiments, these two operational units 805, 810 
may be combined into an integrated operational unit or distributed and executed 
about multiple processors. Further, the image data and model(s) 170 may be stored 
in the local database 135, as shown, or stored in different databases for size, 
security, or administrative reasons. 

20 Figs. 9-12 are fingerprint images 900 embodied in high-resolution image 

data 160 representing the fingerprint 115. Because the processing described herein 
is robust, the image data 160 may be lower resolution than illustrated and still 
achieve the desired matching results. In addition, the gradient edge detection 
processes described herein can accomplish the modeling and comparing processes at 

25 poor contrast levels without significant degradation to the results. However, the user 
may choose to lower a matching threshold at either outline or details levels to 
account for poor contrast levels in the original fingerprint image. 

Figs. 13-20 are illustrative flow diagrams. The flow diagrams may be varied 
in any manner that accomplishes the processing and storage tasks suitable for 

30 achieving the acquisition, modeling, and verification aspects according to the 
principles of the present invention. 
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CLAIMS 

What is claimed is: 

1 . A method for processing an image of a biometric comprising: 
5 applying a gradient edge detection process to detect features in a 

biometric based on data representing an image of at least a portion of the 
biometric; and 

modeling the image as a function of the features. 

10 2. The method according to claim 1 wherein modeling the image includes 
constructing the model for at least two resolutions. 

3. The method according to claim 2 wherein the resolutions include an outline 
model at a low resolution and a details model at a high resolution. 

15 

4. The method according to claim 3 wherein the biometric is an area of skin 
with a friction ridge pattern, the outline model includes edge topology of 
ridge features, and the details model includes edge topology and specifics of 
ridge deviations and locations and sizes of pores. 

20 

5. The method according to claim 2 wherein the resolutions include an outline 
model at a low resolution, a details model at a high resolution, and a fine 
details model used to locate and define particular biometric features more 
accurately than at the low or high resolutions. 

25 

6. The method according to claim 2 wherein the biometric is an area of skin 
with a friction ridge pattern and constructing the model includes identifying, 
outlining, and extracting ridge deviation detail and pore features. 
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7. The method according to claim 6 wherein the ridge deviation detail includes 
ridge contours including scars; and the pore features include position, shape, 
and sizes of pores. 

8. The method according to claim 2 wherein the biometric includes at least one 
of the following: ear shape and structure, facial or hand thermograms, iris or 
retina structure, handwriting, fingerprints, palm prints, foot prints, toe prints. 

9. The method according to claim 1 wherein a gradient is estimated for each 
pixel of the model after applying a noise filter to the image. 

10. The method according to claim 1 wherein the detection process includes 
using a finite differences process. 

1 1 . The method according to claim 1 wherein the detection process includes: 

after calculating the gradients, identifying and marking an image 
point as an edge point having a locally maximal gradient in the direction of 
the gradient that exceeds a threshold; 

identifying neighboring edge points by finding nearest pixels to the 
original edge point that lie in a direction that is approximately perpendicular 
to the gradient direction that passes through the first edge point; 

for the nearest pixels, determining gradient values and, for the pixel 
with a gradient that is maximal along its gradient direction and has a value 
that exceeds a threshold, assigning the pixel to be the next edge point; 

continuing either until the edge is terminated or the edge closes with 
itself to form a continuous curve; 

terminating the process at the previously determined edge point if the 
gradient of the candidate edge point is less than the threshold; and 

repeating the process until all potential edge points have been 
considered. 
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12. The method according to claim 1 further including automatically 
distinguishing biometric features from noise. 

13. The method according to claim 1 2 further including characterizing the noise 
5 as a biometric feature that is below a minimum width or extends less than a 

minimum distance. 



14. The method according to claim 1 further including supporting manual editing 
of features and selecting features that must be present for a successful match. 

10 

15. The method according to claim 1 wherein modeling includes modeling at 
least one region of the image of the at least one portion of the biometric. 

1 6. The method according to claim 1 5 wherein modeling includes overlapping 
15 portions of multiple regions. 

1 7. The method according to claim 15 wherein modeling includes allowing a 
user to add, extend, or delete features. 



20 18. The method according to claim 1 5 wherein modeling includes allowing a 
user to identify features that must be present for a match. 

19. The method according to claim 15 wherein modeling includes allowing the 
user to adjust the size or position of the model relative to the biometric. 

25 

20. The method according to claim 1 wherein the image is a previously stored 
image; and modeling includes normalizing a size of the image as a function 
of a scale associated with the image. 

30 21. The method according to claim 1 wherein the biometric is an area of skin 
with a friction ridge pattern and the features include ridge structure with 
ridge deviation detail. 
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22. The method according to claim 1 further including displaying the image to a 
user with an overlay of indications of the biometric features of the image. 

5 23. The method according to claim 1 further including displaying the image to a 
user with an overlay of indications of filtered biometric features according to 
a selectable criteria. 

24. The method according to claim 1 further including automatically rotating the 
1 0 image to a specified orientation for displaying to a user. 

25. The method according to claim 1 wherein the image is a gray-scale image. 

26. The method according to claim 1 further including adding the image of the at 
1 5 least a portion of the biometric to a database. 

27. The method according to claim 26 wherein adding the image includes storing 
the image and the model of the image in the database. 

20 28. The method according to claim 26 wherein adding the image includes storing 
the image at full sampled resolution. 

29. The method according to claim 26 wherein adding the image includes 
compressing the data representing the image prior to storing in the database. 

25 

30. The method according to claim 29 wherein compressing the image includes 
compressing the image in a lossless manner. 

3 1 . The method according to claim 26 wherein adding the image includes 
30 compressing the model prior to storing the image in the database. 
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30 



32. The method according to claim 26 wherein adding the image includes 
encrypting the data or model prior to storing the image in the database. 

33. The method according to claim 26 wherein adding the image includes storing 
the image and the model of the at least a portion of the biometric with 
associated information. 

34. The method according to claim 33 wherein the associated information 
includes at least one of: identity of a person associated with the biometric; 
manufacturer, model, or serial number of the instrument supplying the data 
representing the portion of the biometric; date of biometric imaging; time of 
day of biometric imaging; calibration data associated with the instrument 
used to acquire the image; temperature at the time the image was acquired; 
unique computer ID receiving the data representing the image from the 
instrument acquiring the image of the biometric; or name of person logged 
into the computer at the time the image was acquired. 

35. The method according to claim 33 wherein the associated information 
includes a photograph, voice recording, or signature of the person whose 
biometric is imaged. 

36. The method according to claim 33 wherein the associated information is a 
watermark. 

37. The method according to claim 36 wherein the watermark is identifying 
information. 

38. The method according to claim 36 wherein the watermark includes anti- 
tampering information. 

39. The method according to claim 1 further including comparing a previously 
stored model from a database to a present image. 
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40. The method according to claim 39 wherein the present image is at least a 
portion of a biometric of a person having a known identity. 

41 . The method according to claim 39 wherein the present image is at least a 
portion of a biometric of a person having an unknown identity. 

42. The method according to claim 39 wherein the present image is received 
from one of the following sources: live source, local database, scanned 
image, or other source. 

43. The method according to claim 39 wherein comparing includes comparing 
outline features of the previously stored model to outline features of the 
present image to determine (i) whether the present image is a candidate for a 
match or (ii) whether the previously stored model is a candidate for a match. 

44. The method according to claim 43 wherein comparing includes determining 
whether the comparison exceeds a predetermined candidate threshold. 

45. The method according to claim 43 wherein, if the present image is not a 
candidate for a match, comparing includes comparing outline features of 
another previously stored model to the outline features of the present image 
to determine whether the present image is a candidate for a match and if so, 
using the next previously stored model for details comparison. 

46. The method according to claim 43 wherein, if the previously stored model is 
not a candidate for a match, comparing includes comparing outline features 
of a next previously stored model to the outline features of the present image 
to determine whether the next previously stored model is a candidate for a 
match and if so, using the next previously stored model for details 
comparison. 
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47. The method according to claim 43 wherein, if a candidate match of outline 
features is found, comparing includes comparing details features of the 
previously stored model with details features of the present image. 

5 48. The method according to claim 47 wherein comparing details features 
includes determining whether the details comparison exceeds a 
predetermined threshold. 

49. The method according to claim 47 wherein comparing details features 
10 includes determining whether required features associated with the 

previously stored model are found in the present image. 

50. The method according to claim 47 wherein the biometric is an area of skin 
with a friction ridge pattern and wherein comparing includes comparing fine 

1 5 details features, Wherein comparing fine details features includes determining 

whether pore features in the previously stored model are found in the present 
image. 

5 1 . The method according to claim 50 further including indicating which pores 
20 in the previously stored model appear in expected locations in the present 

image, including allowing for distortions that normally occur between 
successive impressions. 



52. The method according to claim 5 1 further indicating a pore count or a 
25 statistical probability of an error in at least cases allowing for distortions. 

53. The method according to claim 47 wherein comparing details features 
includes determining whether the details comparison exceeds a 
predetermined threshold a specified number of consecutive frames. 

30 

54. The method according to claim 53 wherein the details features in successive 
frames are different. 
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55. The method according to claim 53 further including selecting another details 
feature set of the previously stored model for correlating with another details 
feature set of the present image. 

5 

56. The method according to claim 53 further including: (i) selecting another 
previously stored model for correlating with a features set of the present 
image and (ii) declaring a successful match if any model exceeds a 
predetermined threshold. 

10 

57. The method according to claim 39 wherein comparing includes comparing 
outline features of the previously stored model to outline features of a model 
of the present image and, if the comparison exceeds a predetermined 
threshold, comparing details features of the previously stored model to 

1 5 details features of the model of the present image to determine whether the 

previously stored model and the present image match. 

58. The method according to claim 39 wherein comparing includes scaling the 
previously stored model, present image, or model of the present image. 

20 

59. The method according to claim 39 wherein comparing includes rotating the 
previously stored model, present image, or present model. 

60. The method according to claim 39 wherein comparing includes adaptively 
25 conforming the previously stored model to account for variability associated 

with recording or acquiring the present image. 

6 1 . The method according to claim 60 wherein accounting for variability 
includes accounting for an expected location of predefined features. 

30 

62. The method according to claim 60 wherein the variability includes stretching 
of the biometric or portions thereof laterally, longitudinally, or radially. 
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63. The method according to claim 60 wherein the variability is caused by 
pressure of the biometric on a medium used to record or acquire the present 
image. 

64. The method according to claim 39 wherein comparing includes comparing 
the previously stored model against multiple present images until a match is 
found or comparison with the multiple present images is complete. 

65. The method according to claim 39 wherein comparing includes comparing 
multiple previously stored models against the present image until a match is 
found or comparison with the multiple previously stored models is complete. 

66. The method according to claim 39 wherein comparing includes comparing 
multiple previously stored models against multiple present images until 
comparing against the multiple previously stored models is complete. 

67. The method according to claim 39 wherein the present image includes 
multiple biometrics of an individual. 

68. The method according to claim 39 wherein the multiple biometrics are 
different areas of skin with a friction ridge pattern from the same individual. 

69. The method according to claim 1 further including comparing previously 
stored models of multiple biometrics to present images of multiple, 
respective biometrics. 

70. The method according to claim 69 further including determining a combined 
metric based on the comparisons. 

71 . The method according to claim 1 further including preprocessing the data 
representing the image. 
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72. The method according to claim 71 further including subsampling the at least 
a portion of the biometric to produce the data representing the image. 

5 73. The method according to claim 71 wherein preprocessing includes 
decimating the data representing the image. 

74. The method according to claim 71 wherein preprocessing includes binning 
the data representing the image. 

10 

75. The method according to claim 71 wherein preprocessing includes correcting 
for uneven imaging of the at least a portion of the biometric. 

76. The method according to claim 7 1 wherein preprocessing includes 

1 5 accounting for defective pixels of an instrument used to acquire the at least a 

portion of the biometrics. 

77. The method according to claim 71 wherein preprocessing includes 
encrypting the data representing the image. 

20 

78. The method according to claim 71 wherein preprocessing includes changing 
the image orientation, including changing the image orientation by flipping 
the image vertically or horizontally or by rotating the image. 

25 79. The method according to claim 71 wherein preprocessing includes attaching 
sensor information to the data representing the image. 



80. 

30 



The method according to claim 71 wherein preprocessing includes applying 
a watermark to the data representing the image. 
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The method according to claim 80 wherein the watermark includes 
information used for tamper-proofing the image to allow for identifying a 
modified image or modified information associated with the image. 



82. An apparatus for processing an image of a biometric, comprising: 

a gradient edge detector that detects features in a biometric based on 
data representing an image of at least a portion of the biometric; and 
a modeler that models the image as a function of the features. 

83. The apparatus according to claim 82 wherein the modeler constructs the 
model for at least two resolutions. 

84. The apparatus according to claim 83 wherein the resolutions include 
an outline model at a low resolution and a details model at a high 
resolution 

85. The apparatus according to claim 84 wherein the biometric is an area of skin, 
with a friction ridge pattern and the outline model includes edge topology of 
ridge features, and the details model includes edge topology and specifics of 
ridge deviations and locations and sizes of pores. 

86. The apparatus according to claim 83 wherein the resolutions include an 
outline model at a low resolution, a details model at a high resolution, and a 
fine details model used to locate and define particular biometric features 
more accurately than at the low or high resolutions. 



87. 

30 



The apparatus according to claim 83 wherein the biometric is an area of skin 
with a friction ridge pattern and the modeler includes an identifier, outliner, 
and extractor to determine ridge deviation detail and pore features. 
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88. The apparatus according to claim 87 wherein the ridge deviation detail 
includes ridge contours including scars; and the pore features include 
position, shape, and sizes of pores. 



5 89. The apparatus according to claim 83 wherein the biometric includes at least 
one of the following: ear shape and structure, facial or hand thermograms, 
iris or retina structure, handwriting, fingerprints, palm prints, or toe prints. 

90. The apparatus according to claim 82 wherein a gradient is estimated for 
10 each pixel of the model after applying a noise filter to the image. 

91 . The apparatus according to claim 82 wherein the detector includes a finite 
differences processor. 



15 92. The apparatus according to claim 82 wherein the detector: 

after calculating the gradients, identifies and marks an image point as 
an edge point having a locally maximal gradient in the direction of the 
gradient that exceeds a threshold value; 

identifies neighboring edge points by finding nearest pixels to the 
20 original edge point that lie in a direction that is approximately perpendicular 

to the gradient direction that passes through the first edge point; 

for the nearest pixels, determines gradient values and, for the pixel 
with a gradient that is maximal along its gradient direction and has a value 
that exceeds a threshold, assigns the pixel to be the next edge point; 
25 continues either until the edge is terminated or the edge closes with 

itself to form a continuous curve; 

terminates the process at the previously determined edge point if the 
gradient of the candidate edge point is less than the threshold; and 
repeats the process until all potential edge points have been 
30 considered. 
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93. The apparatus according to claim 82 wherein the detector automatically 
distinguishes biometric features from noise. 

94. The apparatus according to claim 93 wherein the detector characterizes noise 
5 as a biometric feature that is below a minimum width or extends less than a 

minimum distance. 

95. The apparatus according to claim 82 further including a manual editing 
mechanism in communication with the detector to support manual editing of 

10 features and selecting of features that must be present for a successful match. 

96. The apparatus according to claim 82 wherein the modeler models at least one 
region of the image of the at least one portion of the biometric. 

1 5 97. The apparatus according to claim 96 wherein the modeler models 
overlapping portions of multiple regions. 

98. The apparatus according to claim 96 wherein the modeler allows a user to 
add, extend, or delete features. 

20 

99. The apparatus according to claim 96 wherein the modeler allows a user to 
identify features that must be present for a match. 

1 00. The apparatus according to claim 96 wherein the modeler allows the user to 
25 adjust the size or position of the model relative to the biometric. 

101. The apparatus according to claim 82 wherein the image is a previously stored 
image; and the modeler normalizes a size of the image as a function of a 
scale associated with the image. 
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102. The apparatus according to claim 82 wherein the biometric is an area of skin 
with a friction ridge pattern and the features include ridge structure with 
ridge deviation detail. 

1 03 The apparatus according to claim 7 1 further including a display in 

communication with the modeler that displays the image to a user with an 
overlay of indications of the biometric features of the image. 

104. The apparatus according to claim 71 further including a display in 
communication with the modeler that displays the image to a user with an 
overlay of indications of filtered biometric features according to a selectable 
criteria. 

1 05. The apparatus according to claim 82 further including an image manipulator 
that automatically rotates the image to a specified orientation for displaying 
to a user. 

1 06. The apparatus according to claim 82 wherein the image is a gray-scale 
image. 

107. The apparatus according to claim 82 further including a database that stores 
the image of the at least a portion of the biometric. 

108. The apparatus according to claim 107 wherein the database stores the image 
and the model of the image. 

109. The apparatus according to claim 107 wherein the database stores the image 
at full sampled resolution. 



30 110. 



The apparatus according to claim 107 further including a compressor that 
compresses the data representing the image prior to storing in the database. 
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111. The apparatus according to claim 1 1 0 wherein the compressor compresses 
the image in a lossless manner. 

1 12. The apparatus according to claim 107 wherein the compressor compresses 
5 the model prior to storing the image in the database. 

113. The apparatus according to claim 1 07 further including an encryption unit 
that encrypts the data or model prior to storing the image in the database. 

10 114. The apparatus according to claim 107 wherein the database stores the image 
and the model of the at least a portion of the biometric with associated 
information. 



The apparatus according to claim 1 14 wherein the associated information 
includes at least one of: identity of a person associated with the biometric; 
manufacturer, model, or serial number of the instrument supplying the data 
representing the portion of the biometric; date of imaging the biometric; time 
of day of imaging the biometric; calibration data associated with the 
instrument used to acquire the image; temperature at the time of acquiring 
the image; photograph, voice recording, or signature of the person whose 
biometric is imaged; watermark; unique computer ID of the computer 
receiving the data representing the image from the instrument acquiring the 
image of the biometric; or name of person logged into the computer at the 
time of acquiring the image. 

The apparatus according to claim 1 14 wherein the associated information 
includes a photograph, voice recording, or signature of the person whose 
biometric is imaged. 



30 117. 



The apparatus according to claim 114 wherein the associated information is a 
watermark. 
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118. The apparatus according to claim 1 1 7 wherein the watermark is identifying 
information. 

119. The apparatus according to claim 1 1 7 wherein the watermark includes anti- 
5 tampering information. 

120. The apparatus according to claim 82 further including an input unit that 
provides a present image, a database that stores previously acquired images 
and associated models, and a compare unit that compares a previously stored 

1 0 model from a database to a present image. 

121. The apparatus according to claim 1 20 wherein the present image is at least a 
portion of a biometric of a person having a known identity. 

15 122. The apparatus according to claim 1 20 wherein the present image is at least a 
portion of a biometric of a person having an unknown identity. 

123. The apparatus according to claim 1 20 wherein the present image is received 
from one of the following sources: live source, local database, image 

20 scanner, or other source. 

124. The apparatus according to claim 120 wherein the compare unit compares 
outline features of the previously stored model to outline features of the 
present image to determine (i) whether the present image is a candidate for a 

25 match or (ii) whether the previously stored model is a candidate for a match. 

125. The apparatus according to claim 124 wherein the compare unit determines 
whether the comparison exceeds a predetermined candidate threshold. 



30 126. 



The apparatus according to claim 124 wherein, if the present image is not a 
candidate for a match, the compare unit compares outline features of a next 
previously stored model to the outline features of the present image to 
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determine whether the present image is a candidate for a match and if so, 
uses the next previously stored model for details comparison. 

The apparatus according to claim 124 wherein, if the previously stored 
model is not a candidate for a match, the compare unit compares outline 
features of a next previously stored model to the outline features of the 
present image to determine whether the next previously stored model is a 
candidate for a match and if so, uses the next previously stored model for 
details comparison. 

The apparatus according to claim 1 24 wherein, if a candidate match of 
outline features is found, the compare unit compares details features of the 
previously stored model with details features of the present image. 

The apparatus according to claim 128 wherein the compare unit further 
determines whether the details comparison exceeds a predetermined 
threshold. 

The apparatus according to claim 128 wherein the comparison unit further 
determines whether required features associated with the previously stored 
model are found in the present image. 

The apparatus according to claim 128 wherein the biometric is an area of 
skin with a friction ridge pattern and the comparison unit compares details 
features, wherein the comparison unit determines whether pore features in 
the previously stored model are found in the present image. 

The apparatus according to claim 131 wherein the detector indicates which 
pores in the previously stored model appear in expected locations in the 
present image, the detector allowing for distortions that normally occur 
between successive impressions. 
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133. The apparatus according to claim 1 32 wherein the detector further indicates a 
pore count or a statistical probability of an error in at least cases allowing for 
distortion. 

5 1 34. The apparatus according to claim 128 wherein the comparison unit further 
determines whether the details comparison exceeds a predetermined 
threshold a specified number of consecutive frames. 

135. The apparatus according to claim 1 34 wherein the details features in 
10 successive frames are different. 

1 36. The apparatus according to claim 1 34 wherein the comparison unit selects 
another details feature set of the previously stored model for correlating with 
another details feature set of the present image. 

15 

1 37. The apparatus according to claim 1 34 wherein the comparison unit (i) selects 
another previously stored model for correlating with a feature set of the 
present image and (ii) declares a successful match if any model exceeds a 
predetermined threshold. 

20 

1 38. The apparatus according to claim 120 wherein the comparison units 
compares outline features of the previously stored model to outline features 
of a model of the present image and, if the comparison exceeds a 
predetermined threshold, compares details features of the previously stored 

25 model to details features of the model of the present image to determine 

whether the previously stored model and the present image match. 

1 39. The apparatus according to claim 120 wherein the comparison unit scales the 
previously stored model, present image, or model of the present image. 



140. 



The apparatus according to claim 120 wherein the comparison unit rotates 
the previously stored model, present image, or present model. 
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The apparatus according to claim 120 wherein the comparison unit 
adaptively conforms the previously stored model to account for variability 
associated with recording or acquiring the present image. 

The apparatus according to claim 141 wherein the comparison unit accounts 
for variability by accounting for an expected location of predefined features. 

The apparatus according to claim 141 wherein the variability includes 
stretching of the biometric or portions thereof laterally, longitudinally, or 
radially. 

The apparatus according to claim 141 wherein the variability is caused by 
pressure of the biometric on a medium used to record or acquire the present 
image. 

The apparatus according to claim 120 wherein the comparison unit compares 
the previously stored model against multiple present images until a match is 
found or comparison with the multiple present images is complete. 

The apparatus according to claim 120 wherein the comparison unit compares 
multiple previously stored models against the present image until a match is 
found or comparison with the multiple previously stored models is complete. 

The apparatus according to claim 120 wherein the comparison unit compares 
multiple previously stored models against multiple present images until 
comparing the multiple previously stored models is complete. 



148. 
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The apparatus according to claim 120 wherein the present image includes 
multiple areas of skin with a friction ridge pattern of an individual. 
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149. The apparatus according to claim 120 wherein the multiple biometrics are 
different areas of skin with a friction ridge pattern from the same individual. 

1 50 The apparatus according to claim 82 further including a comparison unit that 
5 compares previously stored models of multiple biometrics to present images 

of multiple, respective biometrics. 

151. The apparatus according to claim 1 50 wherein the comparison unit further 
determines a combined metric based on the comparisons. 

10 

1 52. The apparatus according to claim 82 further including a preprocessor in 
communication with the detector that preprocesses the data representing the 
image. 

15 153. The apparatus according to claim 1 52 wherein the preprocessor subsamples 
the at least a portion of the biometric to produce the data representing the 
image. 

1 54. The apparatus according to claim 1 52 wherein the preprocessor decimates 
20 the data representing the image. 

1 55. The apparatus according to claim 1 52 wherein the preprocessor bins the data 
representing the image. 

25 1 56. The apparatus according to claim 1 52 wherein the preprocessor corrects for 
uneven imaging of the at least a portion of the biometric. 



30 



157. 



The apparatus according to claim 152 wherein the preprocessor accounts for 
defective pixels of an instrument used to acquire the at least a portion of the 
biometric. 
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1 58. The apparatus according to claim 1 52 wherein the preprocessor encrypts the 
data representing the image. 

159. The apparatus according to claim 1 52 wherein the preprocessor changes the 
image orientation by flipping the image vertically or horizontally or by 
rotating the image. 

1 60. The apparatus according to claim 1 52 wherein the preprocessor attaches 
sensor information to the data representing the image. 

161. The apparatus according to claim 1 52 wherein the preprocessor applies a 
watermark to the data representing the image. 

162. The apparatus according to claim 161 wherein the watermark includes 
information used for tamper-proofing the image to allow for identifying a 
modified image or modified information associated with the image. 

163. An apparatus for processing an image of a biometric, comprising: 

gradient edge detection means for detecting features in a biometric 
based on data representing an image of at least a portion of the biometric; 
and 

means for modeling the image as a function of the features. 

164. A method for processing an image of a biometric, comprising: 

acquiring data representing an image of at least a portion of a 
biometric; and 

modeling features of the biometric for at least two resolutions. 

165. The method according to claim 1 64 wherein the resolutions include an 
outline model at a low resolution, a details model at a high resolution, and a 
fine details model used to locate and define particular biometric features 
more accurately than at low or high resolutions. 
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166. The method according to claim 164 wherein the biometric is an area of skin 
with a friction ridge pattern, the outline model includes edge topology of 
ridge features, and the details model includes edge topology and specifics of 
ridge deviations and locations and sizes of pores. 

167. The method according to claim 164 wherein modeling the image includes 
applying a gradient edge detection process. 

168. The method according to claim 164 wherein the biometric includes at least 
one of the following: ear shape and structure, facial or hand thermograms, 
iris or retina structure, handwriting, fingerprints, palm prints, foot prints,or 
toe prints. 

1 69. The method according to claim 164 further including storing the image and 
model in a database with associated information, including at least one of the 
following: identity of a person associated with the biometric; manufacturer, 
model, or serial number of the instrument supplying the data representing the 
portion of the biometric; date of imaging the biometric; time day of imaging 
the biometric; calibration data associated with the instrument used to acquire 
the image; temperature at the time of acquiring the image; photograph, voice 
recording, or signature of the person whose biometric is imaged; watermark; 
unique computer ID of the computer receiving the data representing the 
image of the biometric; or name of person logged into the computer at the 
time of acquiring the image. 

1 70. The method according to claim 1 64 further including comparing a previously 
stored model from a database to a present image. 



30 171. 



The method according to claim 170 wherein the present image is at least a 
portion of a biometric of a person having a known identity or having an 
unknown identity. 
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The method according to claim 170 wherein, if a candidate match of outline 
features is found, the method further includes comparing details features of 
the previously stored model with details features of the present image. 

The method according to claim 1 70wherein the comparing includes 
adaptively conforming the previously stored model to account for variability 
associated with recording or acquiring the present image. 

The method according to claim 1 64 further including preprocessing the data, 
wherein the preprocessing includes at least one of the following: 

subsampling the at least a portion of the biometric to produce the data 
representing the image; 

decimating the data representing the image; 

binning the data representing the image; 

correcting for uneven imaging of the at least a portion of the 
biometric; 

accounting for defective pixels of an instrument used to acquire the at 
least a portion of the biometric; 

encrypting the data representing the image; 

changing the image orientation by flipping the image vertically or 
horizontally or by rotating the image; 

attaching sensor information to the data representing the image; or 

applying a watermark to the data representing the image. 

An apparatus for processing an image of a biometric, comprising: 

a data acquisition unit that acquires data representing an image of at 

least a portion of a biometric; and 

a modeler that models features of the biometric for at least two 

resolutions. 
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176. The apparatus according to claim 175 wherein the resolutions include an 
outline model at a low resolution, a details model at a high resolution, and a 
fine details model used to locate and define particular biometric features 
more accurately than at low or high resolutions. 

5 

177. The apparatus according to claim 176 wherein the biometric is an area of 
skin with a friction ridge pattern, the outline model includes edge topology of 
ridge features, and the details model includes edge topology and specifics of 
ridge deviations and locations and sizes of pores. 

10 

178. The apparatus according to claim 175 wherein the modeler includes a 
gradient edge detector to detect biometric features in the image. 

1 79. The apparatus according to claim 1 75 wherein the biometric includes at least 
15 one of the following: ear shape and structure, facial or hand thermograms, 

iris or retina structure, handwriting, fingerprints, palm prints, or toe prints. 

1 80. The apparatus according to claim 175 further including a database that stores 
the image and model with associated information, including at least one of: 
identity of a person associated with the biometric; manufacturer, model, or 

20 serial number of the instrument supplying the data representing the portion of 

the biometric, date of imaging the biometric; time of day of imaging the 
biometric; calibration data associated with the instrument used to acquire the 
image; temperature at the time of acquiring the image; photograph, voice 
recording, or signature of the person whose biometric is imaged; watermark; 

25 unique computer ID of the computer receiving the data representing the 

image from the instrument acquiring the image of the biometric; or name of 
person logged into the computer at the time of acquiring the image. 



181. 

30 



The apparatus according to claim 175 further including a comparison unit 
that compares a previously stored model from a database to a present image. 
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1 82. The apparatus according to claim 1 8 1 wherein the present image is at least a 
portion of a biometric of a person having a known identity or having an 
unknown identity. 

5 183. The apparatus according to claim 1 8 1 wherein, if a candidate match of 

outline features is found, the comparison unit compares details features of 
the previously stored model with details features of the present image. 

1 84. The apparatus according to claim 1 8 1 wherein the comparison unit 

10 adaptively conforms the previously stored model to account for variability 

associated with recording or acquiring the present image. 

1 85. The method according to claim 1 75, further including a preprocessor that 
preprocesses the data, the preprocessor including at least one of the 

15 following components: 



a subsampler that subsamples the at least a portion of the biometric to 



20 



produce the data representing the image; 

a decimator that decimates the data representing the image; 

a binning unit that bins the data representing the image; 

a field flattener that corrects for uneven imaging of the at least a 



portion of the biometric; 

a defective pixel correction unit that accounts for defective pixels of 



25 



an instrument used to acquire the at least a portion of the biometric; 

an encryption unit that encrypts the data representing the image; 
an image orientation unit that changes the orientation by flipping the 



image vertically or horizontally or by rotating the image; 

a sensor data application unit that attaches sensor information to the 



data representing the image; or 

a watermark application unit that applies a watermark to the data 



30 



representing the image. 
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A method for processing an image of a biometric, comprising: 

means for acquiring data representing an image of at least a portion 

of a biometric; and 

means for modeling features of the biometric for at least two 

resolutions. 
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Fingerprint Enrollment 
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Transfer Image to local or remote computer 



Convert image lo desired format, add header informal 



Construct a Trained Model (s) 



Compress and encrypt (optional) both image and model 



Save Image and fingerprint model in data base 
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Construct a "trained model y^ 0 * 

Construct a model for at least two resolution levels; a low resolution "outline" model and a high 
resolution "details" model 

J T 

Choose one or more regions of the fingerprint image to use for the model 




In eachtif the selected regions of the fingerprint ana at each of the chosen resolutions 
use gray-level, gradient edge detection techniques 
to identify, outline, and extract ridge profiles and pore features 



ijao 



Optional: Manually add, extend, or delete features j 
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Optional: Identify features that must be present for a match 
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EXHIBIT D 



Vincent Fedele 

P.O. Box 061 
Harvard, MA 01451-0061 
U.S.A. 



27 July, 2006 

Beyer Weaver & Thomas, LLP 
500 12 th Street 
Suite 200 

Oakland, CA 94607 

Subj : Docket No. 3174.1012-013 



Dear Mr. Egan: 

I am in receipt of your letter of 20 July, 2006 regarding the above-mentioned docket. 

As you may be aware, I left Aprilis, Inc. on February 03, 2005. One-month prior on or 
about January 3 rd I was provided with assignment and declaration documents by David 
Waldman for Docket No. 3174.1012-013, and took them immediately to Citizens Bank in 
Maynard, whose offices are on Nason Street, just outside of the Aprilis location. The 
documents were signed and notarized, and returned to David Waldman by hand in time to 
meet the mailing date for this application. 

During several conversations with various officers of Aprilis during the week after my 
departure in February, Aprilis and Mr. Julio Vega, Counsel for Aprilis, confirmed that all 
assignments and declarations had been properly executed and received by the Company. 

Any pertinent prior art, which I may have been aware of during the period I was 
employed by Aprilis, was disclosed at the time, and I am not aware of any other prior art 
for 3174.1012-013 since January of 2005. 

An email message from Mr. Waldman in April or May of this year referred to several 
changes and or challenges made by the patent officer reviewing the file(s) that would 
require changes to one or both of the applications pending review. References to an other 
patent may also have been included, Docket No. 3174.1013-14. Neither patent 
application was provided, however, only assignment and declaration pages in Mr. 
Waldman' s email message. 

Since the assignment and or declaration documents, as well as your letter include 
statements that indicate I must read, review and understand the contents of the 
application(s), and since it has been several years since I have worked on these 
inventions, I will require at least a day or two to review them, and possibly some time 



over the phone with Mr. Waldman. Please clarify if Docket No. 3 1 74. 1 0 1 3- 1 4 is also in 
need of assignment, declaration and requisite review, as it was not included in your 
package, but was mentioned in Mr. Waldman's email. 

I am happy to provide such review(s), time permitting, as a consultant to the Company, 
and in accordance with my agreement with the Company which included a provision for 
compensation for such tasks. I estimate approximately one day, at a minimum, will be 
required to review the patent application you sent which appears to include more than one 
hundred claims and some seventy-five pages of text, figures and related documents. 

I am planning travel and vacation during the next 2-3 weeks, if I do not hear back from 
you by August 8 th , I will return the document package to your offices. 



Vincent Fedele 



5. Inventions; Assignment . 



(d) At the request of the Company, the Signatory will assist the 
Company in every proper way (including, without limitation, by 
executing patent applications) to obtain and enforce in any country 
in the world Proprietary Rights relating to any or all Assigned 
Inventions. The Signatory's obligation under this Section 5(d) shall 
continue after the termination of the Signatory's association with the 
Company as an employee, consultant, officer or director. If and to 
the extent that, at any time after the termination of the Signatory's 
association with the Company as an employee, consultant, officer 
and/or director, the Company requests assistance from the Signatory 
with respect to obtaining and enforcing in any country in the world 
any Proprietary Rights relating to Assigned Inventions, the 
Company shall compensate the Signatory at a reasonable rate for 
the time actually spent bv the Signatory on such assistance . 



EXHIBIT E 



t United States Patent and Trademark Office 



Commissioner for Patents 
United States Patent and Trademark Office 
P.O. Box 1450 
Alexandria, VA 22313-1450 



Paper No: _ 



BEYER WEAVER LLP 
P.O. BOX 70250 

OAKLAND CA 94612-0250 COPY MAILED 



In re Application of \^\C^ Q^OO \ 
Waldman, et al. 
Application No. 10/561,646 
Filed: 20 December, 2005 
Attorney Docket No. 3 1 74. 1 0 1 2-0 1 3 



MAR 0 2 2007 
OFFICE OF PETITIONS 

ON PETITION 



This is a decision on the petition filed on 16 October, 2006, under 37 C.F.R. §1.47. 
For the reasons set forth below, the petition under 37 C.F.R. §1.47 is GRANTED . 

BACKGROUND 

The record indicates: 

the instant application was filed on 20 December, 2005, without, inter alia, a fully 
executed oath/declaration; 

on 21 March, 2006, the Office mailed a Notice of Missing Parts indicating, inter alia, that 
a fully executed oath/declaration (signed and dated) was required; 

on 1 6 October, 2006, Petitioner William J. Egan III (Reg. No. 28,41 1) filed, inter alia, the 
original petition, an oath/declaration signed by co-inventors Waldman, Ingwall, Raguin, 
Berg, Joseph and Kent for themselves and on behalf of the nonsigning inventor Vincent 
Fedele (Mr. Fedele) and an averment, supported by a copy of the transmittal, that the 
entire application (description, claims, abstract and drawings) as required by statute and 
regulation, was sent to Mr. Fedele, and an averment as to the reasonably believed to be 
last known/current mailing address for Mr. Fedele-and while Mr. Fedele' s reply o 
July, 2006, refers to more than one application, it specifically cites the attorney d 
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number (3174.1012-013) of the instant application and on the second page specifically 
acknowledges receipt of the instant application. 

Out of an abundance of caution, Petitioners always are reminded that those registered to practice 
and all others who make representations before the Office are reminded to inquire into the 
underlying facts of representations made to the Office and support averments with the 
appropriate documentation-since all owe to the Office the continuing duty to disclose. 1 

This application and its papers have been reviewed and found in compliance with 37 C.F.R. 
§1.47(a). 



See supplement of 1 7 June, 1999. The Patent and Trademark Office is relying on petitioner's duty of candor and good faith and 
accepting a statement made by Petitioner. See Changes to Patent Practice and Procedure. 62 Fed. Reg, at 53 1 60 and 53 1 78, 1 203 Off. Gaz. Pat. 
Office at 88 and 103 (responses to comments 64 and 109)(applicant obligated under 37 C.F.R. §10.18 to inquire into the underlying facts and 
circumstances when providing statements to the Patent and Trademark Office). 

Specifically, the regulations at 37 C.F.R. §10.18 provide: 

§ 10.18 Signature and certificate for correspondence filed in the Patent and Trademark Office. 

(a) For all documents filed in the Office in patent, trademark, and other non-patent matters, except for correspondence that is required to be 
signed by the applicant or party, each piece of correspondence filed by a practitioner in the Patent and Trademark Office must bear a signature 
by such practitioner complying with the provisions of § 1.4(d), § 1.4(e), or § 2.193(c)(1) of this chapter. 

(b) By presenting to the Office (whether by signing, filing, submitting, or later advocating) any paper, the party presenting such paper, whether a 
practitioner or non-practitioner, is certifying that— 

(1) All statements made therein of the party's own knowledge are true, all statements made therein on information and belief are 
believed to be true, and all statements made therein are made with the knowledge that whoever, in any matter within the jurisdiction of the Patent 
and Trademark Office, knowingly and willfully falsifies, conceals, or covers up by any trick, scheme, or device a material fact, or makes any 
false, fictitious or fraudulent statements or representations, or makes or uses any false writing or document knowing the same to contain any 
false, fictitious or fraudulent statement or entry, shall be subject to the penalties set forth under 1 8 U.S.C. 1 001 , and that violations of this 
paragraph may jeopardize the validity of the application or document, or the validity or enforceability of any patent, trademark registration, or 
certificate resulting therefrom; and 

(2) To the best of the party's knowledge, information and belief, formed after an inquiry reasonable under the circumstances, that — 

(i) The paper is not being presented for any improper purpose, such as to harass someone or to cause unnecessary delay or needless 
increase in the cost of prosecution before the Office; 

(ii) The claims and other legal contentions therein are warranted by existing law or by a nonfrivolous argument for the extension, 
modification, or reversal of existing law or the establishment of new law; 

(iii) The allegations and other factual contentions have evidentiary support or, if specifically so identified, are likely to have 
evidentiary support after a reasonable opportunity for further investigation or discovery; and 

(iv) The denials of factual contentions are warranted on the evidence, or if specifically so identified, are reasonably based on a lack of 
information or belief. 

(c) Violations of paragraph (b)(1) of this section by a practitioner or non-practitioner may jeopardize the validity of the application or document, 
or the validity or enforceability of any patent, trademark registration, or certificate resulting therefrom. Violations of any of paragraphs (b)(2)(i) 
through (iv) of this section are, after notice and reasonable opportunity to respond, subject to such sanctions as deemed appropriate by the 
Commissioner, or the Commissioner's designee, which may include, but are not limited to, any combination of — 

(1) Holding certain facts to have been established; 

(2) Returning papers; 

(3) Precluding a party from filing a paper, or presenting or contesting an issue; 

(4) Imposing a monetary sanction; 

(5) Requiring a terminal disclaimer for the period of the delay; or 

(6) Terminating the proceedings in the Patent and Trademark Office. 

(d) Any practitioner violating the provisions of this section may also be subject to disciplinary action. See § 1 0.23(c)(l 5). 

[Added 50 FR 5175, Feb. 6, 1985, effective Mar. 8, 1985; para, (a) revised, 58 FR 54494, Oct. 22, 1993, effective Nov. 22, 1993; paras, (a) & 
(b) revised, paras, (c) & (d) added, 62 FR 53131, Oct. 10, 1997, effective Dec. 1, 1997; para, (a) revised, 69 FR 56481, Sept. 21, 2004, effective 
Oct. 21, 2004] 



Application No. 10/561 ,646 



3 



This application hereby is ACCORDED status under 37 C.F.R. §1.47(a) . 

As provided under 37 C.F.R. § 1.47(a), the Office will forward notice of this application's filing 
to the non-signing inventor at the address given in the petition. 

Notice of the filing of this application also will be published in the Official Gazette. 

This file is being released to PIPE for processing as necessary to reflect the instant decision 
before being released for examination in due course. 

While telephone inquiries regarding this decision may be directed to the undersigned at (571) 
272-3214, it is noted that all practice before the Office is in writing fsee: 37 C.F.R. S1.2 2 ) and the 
proper authority for action on any matter in this regard are the statutes (35 U.S.C.V regulations 
(37 C.F.R.) and the commentary on polic y (MPEP). Therefore , no telephone discussion may be 
controlling or considered authority for Petitioner WCaller's action( s ) . 



John J. Gillon, Jr. 
Senior Attorney 
Office of Petitions 



The regulations at 37 C.F.R. §1.2 provide: 
§1.2 Business to be transacted in writing. 

All business with the Patent and Trademark Office should be transacted in writing.The personal attendance of applicants or their attorneys or 
agents at the Patent and Trademark Office is unnecessary. The action of the Patent and Trademark Office will be based exclusively on the 
written record in the Office. No attention will be paid to any alleged oral promise, stipulation^ understanding in relation to which there is 
disagreement or doubt. 





Commissioner for Patents 
United States Patent and Trademark Office 
P.O. Box 1450 
Alexandria, VA 22313-1450 



JJGJR. 02-07 



Paper No: 



VINCENT FEDELE 

P.O. box 061 COPY MAILED 

HARVARD, MA 01451 

MAR 0 2 2007 

In re Application of \M^ ?OC: >l OFFICE OF PETITIONS 

Waldman, et al. : 

Application No. 10/561,646 : COMMUNICATION 

Filed: 20 December, 2005 : 

Attorney Docket No. 3 1 74. 1 0 1 2-0 1 3 : 



Dear Vincent Fedele: 

You are named as inventor in the above identified United States patent application, filed under 
the provisions of 35 U.S.C. §116 (United States Code), and 37 C.F.R §1.47,' |a, Rules of 
Practice in Patent Cases. Should a patent be granted on the application you will be designated 
therein as inventor. 

As a named inventor you are entitled to inspect any paper in the file wrapper of the application, 
order copies of all or any part thereof (at a prepaid cost per 37 C.F.R. §1.19) or make your 
position of record in the application. Alternatively, you may arrange to do any of the preceding 



The regulations at 37 C.F.R. §1.47 provide: 
§ 1.47 Filing when an inventor refuses to sign or cannot be reached. 

(a) If ajoint inventor refuses to join in an application for patent or cannot be found or reached after diligent effort, the application may be made 
by the other inventor on behalf of himself or herself and the nonsigning inventor. The oath or declaration in such an application must be 
accompanied by a petition including proof of the pertinent facts, the fee set forth in § 1 . 1 7(h), and the last known address of the nonsigning 
inventor. The nonsigning inventor may subsequently join in the application by filing an oath or declaration complying with §1.63. 

(b) Whenever all of the inventors refuse to execute an application for patent, or cannot be found or reached after diligent effort, a person to 
whom an inventor has assigned or agreed in writing to assign the invention, or who otherwise shows sufficient proprietary interest in the matter 
justifying such action, may make application for patent on behalf of and as agent for all the inventors. The oath or declaration in such an 
application must be accompanied by a petition including proof of the pertinent facts, a showing that such action is necessary to preserve the 
rights of the parties or to prevent irreparable damage, the fee set forth in §1.1 7(h), and the last known address of all of the inventors. An inventor 
may subsequently join in the application by filing an oath or declaration complying with §1 .63. 

© The Office will send notice of the filing of the application to all inventors who have not joined in 
in the petition under this section, and publish notice of the filing of the application in the Official Gazette. 



yt divisional application, if notice regarding the filing of the prior applic tion 
inventors). SEE! 
[47 Fed. Reg. 41275, Sept. 17, 1982, effective Oct. 1, 1982; 48 Fed. Reg. 2709, Jan. 20, 1983, effective F ' " 
53131, Oct. 10, 1997, effective Dec. 1, 1997; revised, 65 Fed. Reg. 54604, Sept. 8, 2000, effective Nov. 7 
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through a registered patent attorney or agent presenting written authorization from you. If you 
care to join the application, counsel of record (see below) would presumably assist you. Joining 
in the application would entail the filing of an appropriate oath or declaration by you pursuant to 
37 C.F.R. §1.63. 

Requests for information regarding your application should be directed to the File Information 
Unit at (703) 308-2733. 

Information regarding how to pay for and order a copy of the application, or a specific paper in 
the application, should be directed to Certification Division at (703) 308-9726 or toll-free: (800) 
972-63 82 (outside the Washington D.C. area). 

While telephone inquiries regarding this decision may be directed to the undersigned at (571) 
272-3214, it is noted that all practice before the Office is in writing (see: 37 C.F.R. S1.2 2 ) and the 
proper authority for action on any matter in this regard are the statutes (35 U.S.C.). regulations 
(37 C.F.R.) and the commentary on policy (MPEP). Therefore, no telephone discussion may be 
controlling or considered authority for Petitioner's/Caller's action(s) . 



Joprr^. Gallon, Jr. 
Senior Attorney 
Office of Petitions 

Counsel of Record: 
BEYER WEAVER LLP 
P.O. BOX 70250 
OAKLAND CA 94612-0250 



2 The regulations at 37 C.F.R. §1.2 provide: 
§1.2 Business to be transacted in writing. 

All business with the Patent and Trademark Office should be transacted in writing.The personal attendance of applicants or their attorneys or 
agents at the Patent and Trademark Office is unnecessary. The action of the Patent and Trademark Office will be based exclusively on the 
written record in the Office. No attention will be paid to any alleged oral promise, stipulation^ understanding in relation to which there is 
disagreement or doubt. 




